Ask HN: What do you use Machine Learning for?
What real-world applications are you finding it useful for right now?
I've found it very effective in the role of "source of general, constant, low-level anxiety".
I don't think ML or AI (or whatever you want to call it) is going to replace programmers any time soon, either. I do think, though, that having the skills in such domains will be something that employers are going to look at in differentiating potential candidates for job placements. So if you have the knowledge or can get it now, it may prove useful in the future.
I find it frustratingly hard to learn ML on my own, when I don't have a real problem to solve, just exercises.
Or - go smaller (probably safer, too!) - make a self-driving RC car!
I saw a quote around here like "Data has gravity; the computation will move to it." (I think this was Nadella at Build.) This is true for ML in a way that was not true for previous computer applications (I mean, mainstream applications of computers). You could write an image filter or an audio effect (or a spreadsheet, or a mail client, etc) with no data at all (or a small amount of ad-hoc test data). But this suggests a subversion in the problem-finding process. Instead of thinking about what problems algorithms could solve, you have to look for large corpuses of data. It does feel like a paradigm shift that should engage our self-interest, if nothing else.
Self driving vehicles - or other robotics localization and similar problems. Those are fun, at least, I find them fun.
I've also considered stuff in the graphics arts - like the various "deep dreaming" things, or the various ML filtering of images (turning a painting into a "photo" for instance); there's some interesting possibilities there.
But for me - robots are where it's at. The idea of making a machine that can "do its own thing" intelligently - well, that has fascinated me ever since I was a small child and watched the original 1970s "Buck Rogers" series with my dad.
"Over the past decades computers have broadly automated tasks that programmers could describe with clear rules and algorithms. Modern machine learning techniques now allow us to do the same for tasks where describing the precise rules is much harder."
Back on point... this is one of those times in my career when I have to say "This changes everything." ML changes everything. How we formulate problems, how we architect solutions, how we program solutions.
For practical advice: I started the Udacity self-driving car Nano-degree. (I won't be finishing it, but that is another story.) The first term is pretty much all ML. I found the Udacity course exactly what I needed to get a practical, hands-on intro and ramp-up to the current world of ML. I'm no expert, but I now at least recognize the vocabulary, have rubbed my brain cells against a few simple ML problems, and can move forward and learn more. Udacity also has a ML nano-degree. YMMV.
I did the CS373 course back in 2012 - and I have found that the nanodegree (which I am taking, and I am about 2/3s of the way thru the second term) touches on more than a bit of new stuff that I didn't find covered in the CS373 course (at least back in 2012).
First term alone is worth the price, I think. Learning how to use OpenCV, TensorFlow and Keras, then using that knowledge to "drive" a virtual car (I built a version of the NVidia End-to-End CNN and trained it using CUDA and my GPU at home, using data I derived in the simulator by steering the virtual car using a steering wheel controller I bought just for that purpose).
Second term so far has felt mostly like a re-hash of what was learned in the CS373 - although, learning about extended and unscented Kalman filters for integrating LIDAR and RADAR data was not something covered in CS373 (only the basic Kalman filter - useful in itself, though).
My goal is to finish this puppy.
From what I have heard (anecdotally), when the initial class started, for the first term there were approximately 30,000 students enrolled. When term 2 started, only about 300 of those students enrolled for the new term.
So far, I'm doing pretty well with this. Considering I don't have an advanced degree, that I am a self-taught software developer, and I am going to have my 44th birthday in about a month - well, you know the story about dogs and tricks; that don't apply to me.
I'd be curious to know why myself?
I'm about 2/3rds of the way thru the second term (I'm in the November cohort), and I plan to finish the nanodegree.
What I have found, though, with this second term, is that it seems to be a rehash of the CS373 course (I took that back in 2012, when Udacity first started) - though it also has taught a few new concepts for me (I like the fact that it is mostly done in C++ - which had gotten really rusty for me - but now I am liking C++ more).
I'd also recommend the Coursera ML course - from what I know, it is basically identical (if not the same material) as the original ML Class from 2011 (I took that one, which was my intro to MOOCs). If you can wrap your mind around the programming language used (Octave - basically an open-source clone of MatLab - where the main primitive is a vector/list), you'll find it does a great job at explaining many concepts that you might have had trouble with in the past (when we got to neural networks and backprop, it finally clicked for me - same with how think in terms to figure out how to parallelize processes).
I like the video lectures a lot, and got a lot out of them. I like the quizes and short problems. The large projects are a little too color-by-numbers for my taste, I really would like to see an incremental exercise step in between in order to flesh out understanding more solidly, but if I had enough time I could probably fill than in myself.
But as I said above, my decision not to follow through is mostly personal circumstances. When I signed up for the course I was casting around for something new to do and thought the course would add to my robotics knowledge and cred. By the time my class cohort started, I was working at a robotics startup. Aside from the startup taking a lot of time and energy, doing robots by day, robots at night, and robots on the weekend just got to be a bit much. Something in my life had to give, and Udacity didn't make the cut this time.
You're right on the "color by numbers" part - but I think there's a couple of reasons for it. I think that the purpose of the course isn't geared only to ML and self-driving vehicle purposes, but also as a means toward general software development. So they don't expect everyone to be at the self-starting coder level.
Secondly, if they were to, say, have you code each project from scratch with only a bare outline of what is needed, I think they'd have to likely expand the course (more lessons to teach things more in depth), as well as extend the time for the lessons; all of this would mean the course would be much longer, and that students would have to dedicate even more time to the effort, beyond what they already are doing.
Some could do that, but I think that many wouldn't be able to; they have to strike a balance somewhere, and this has been their answer. Plus, I have a feeling that many of their students are currently students going to university in some manner, and not all of them in the United States, but abroad internationally. Many of these students are likely struggling just to make the payments for this nanodegree as-is (and may have contributed to the drop-out rate from term 1 to term 2). Making the course longer and more expensive might put it completely out of reach for those students.
Alternative scenario: loss of jobs to automation and inadequate social safety net cause social unrest and communist revolution. We've seen this kind of thing before.
In short: if you're looking at Hacker News then doing ML is likely irrelevant.
Honestly it sounds like you're paranoid about your job security and externalizing it. "It's ML, no it's foreigners, no it's the aliens!" I'm a programmer, not a psychologist, though, so don't take my word for it. You can have a look at the sklearn docs though, good place to start on your ML quest.
http://scikit-learn.org/stable/
[1] Not intended to be a factual claim, just a point about the current political climate
My job security's fine as long as all these dumb "Facebook for Dogs", mid-sized software company doing nothing all that amazing (so: 99% of mid sized software companies), and "Non-software-company software division" jobs that comprise a giant percentage (almost certainly the vast majority) of US development jobs remain in the US (or other expensive OECD states). I'm just not certain why they would remain here, in the medium-term. It's not paranoia so much as "gee, why the hell am I paid so much?" The reasons seem pretty weak, which... isn't comforting. It's not like I'm curled up shaking in the corner over it, though.
I guess my overarching point here is that global economics is one of the more dizzyingly complex things you can choose to worry about so I tend not to.
Let's say you were an automobile assembly line worker in 1965, and you were hearing about robots taking over your job at some point. What would you do? That is, what would be the best thing you could do?
If you answer "Get an education in something that robots aren't doing, and try to get out of this career path" - go to the front of the line.
So - if you are a software developer - and worried that sometime in the future (I'd argue that "sometime" will soon mean "in 5 years, maybe less" - just as you note), ML is "rowing into a required, essential skill for a software developer" - what would you do?
Sit back, throw your hands up, and hope?
Or maybe get that education now on how ML works, how to use it, learning the hard math, etc?
Guess which route I have taken? Perhaps others should as well, if they want to continue in their careers...
Projects like TPOT and DataRobot are already showing a huge amount of promise for automating the tedious/time consuming parts of ML. But this doesn't mean they'll get good at the hard part any time soon.
I'd say your time/energy is better spent lobbying governments for socialist reform than learning ML at this point so we can avoid the worst case scenario(s).
Those who are still in school or fresh out of it will be up-to-date with the latest theory and trends. And they will be eager to pour hours of overtime into their new career. Sure, they will have years of practical experience to gain, but that's not usually what they will be interviewed on (from my experience). They will be interviewed on the latest tech and the algorithms and theory they learned in school. Their whiteboard interviews will be another day from class.
Given the influx of fresh blood, I view the software engineering profession as a entry-level type of job, regardless of the varying levels of seniority titles available. The only way to escape the constant churn is move to other roles, such as architect, management, executive or to start a business such as in niche consulting or a startup, etc. Or you can eschew the typical life of mortgage + kids and dedicate it to keeping up with every latest trend, hopping jobs to stay current and hoping you never slow down.
I wonder what our technology choices would look like if this weren't the case. In other words, suppose someone said to you "you'll be responsible for these software applications for the next 10 years" vs "you'll need to find a new job every 2-3 years for the next 10 years." Would we adopt and apply new technologies this aggressively? Would we be changing javascript frameworks constantly? Would we necessarily be using them at all, vs a wait and see approach? (btw, yes, depending on the project sometimes we would use them, but my guess is that we would far less often).
It's happened a bunch of times in my career by now. Currently, I think our huge over-application of technology is in the realm of constantly evolving front end javascript frameworks. I still believe, firmly, that most smallish apps would be better handled through an integrated framework like rails or Django, with a bit of judiciously applied javascript when necessary. Or, perhaps, even no framework at all.
But unfortunately, that doesn't get you jobs. Almost all the rails jobs out there now require expertise in Ember, React, etc. So the truth is, it's rational to add these things to your current rails project even if you don't think the trade-off is quite worth it.
Just to be clear, yeah, sometimes it is worth it. I look at it this way - for a while there, most land animals had exoskeletons, even some fairly large ones. Then, endoskeletons came along, and it turned out that to scale up to be a giant land animal, you needed an endoskeleton. Some of the larger animals with exoskeletons started to go extinct, as the more scalable endoskeletons drove them out of their ecological niches.
Here's the part that they don't cover on the nature channel - recruiters started to ask giant centipedes if they had any experience with endoskeleton. Senior architects and technology directs started insisting that fleas, ticks, and flying gnats have endoskeletons, because they attended a conference that told them that if you want to to wooly mammoth size, you need endoskeleton.
Meanwhile, the creators of endoskeleton frameworks are (rightly) all like, "Don't put this on me, I never said a gnat should have an endoskeleton".
Oh as for machine learning... yeah, I suppose there may be some use of spark, map reduce, Hadoop, and so forth that exists to gain experience with these technologies rather than because they're needed. That definitely happened with some of the noSql stuff. But honestly, the unnecessary churn on the back end/data science side pales in comparison to what happens out there in the web world.
The theory doesn't change that fast... also trends? Maybe in terms of a new JS library.. but I think it's more of new grads will work for cheap and will take more bullshit. They also are more likely to move to/for the job.
There are also a lot of resources for ML that don't require diving off the deep end such as tutorials/intro classes on coursera. In my opinion also picking what you learn helps as well. Trying to skip straight to neural networks might not be the best place to start if you don't know anything about linear regressions or classifiers.
Semi-related, but if anyone is using the AWS tools for their AI, please ping me. I'm looking for a speaker for a community event in SF in June. (contact info in profile)
I think the problem you'll end up having is that in some/most cases you are never really certain what the problem was.
Feeding might make the crying stop but you have no way of knowing if the reason for the crying was hunger. Similar problems for sleep, diaper changing, etc.
Every time the crying stops you can never be certain if you just made them temporarily forget about the root problem or if you did properly identify the root cause.
TL/DR my hypothesis after raising my own kids, is that the data will be almost all noise with very little perceptible signal.
You won't be able to have anywhere near enough data for that.
Unless somewhere out there is a labeled dataset of newborn baby faces, with around 20-30,000 unique samples (the more data, the better).
Your problem isn't really that difficult; it could probably be done with a very simple neural network a few hours of training. If you wanted to be robust, you could hack the ImageNet CNN into something to do this, too.
I would suggest, though, while what you are doing sounds fun - enjoy being a father and deal with this the old fashioned way. It'll probably be more satisfying.
/heck, what do I know - I don't have kids...
http://www.vocativ.com/414390/neural-network-identifes-porn-...
I didn't develop the original approach but am involved in working to improve the predictive outcomes.
I use machine learning to predict if a newly designed part will have to be purchased soon to go in the lab. That prediction saves me about 3 weeks of lead time to get it to a supplier. Before machine learning, the first time the purchasing department knew that it will need to be purchased is when the designer filled out a form. Now they know as soon as he submits his design and can get a head start.
Does anybody know what the value of such lists are? Having no other information with them I can't imagine it is much.
You'd be making the world a better place by encouraging the act of exercising. And the lower cost of healthcare. It's pretty much up to you now!
We also use machine learning for detecting other features of EEG data and removing artifacts. (Eye blinks, for example, cause big artifacts.)
This will get worse as more retail establishments use image recognition in their every day operation.
https://en.wikipedia.org/wiki/Learning_to_rank
https://cwiki.apache.org/confluence/display/solr/Learning+To...
Did you put an add out on craigslists requesting for people to attempt it or do you have some criminal connections?
Maybe a Restricted Boltzmann Machine or something similar?
We use machine learning to predict the risk of a patient, after an inpatient admission, being readmitted in 30 days post discharge. If the risk is high, we proactively put process in place to reduce the readmission risk.For instance when I get home (gps sensor) and when I open my front door (door binary sensor), can a system understand that I typically turn my TV on when that happens?
Did you evaluate other NER against CoreNLP? e.g. how does spacy compare?
EDIT: answering my own question -- research here [1] ranks CoreNLP as the winner among CoreNLP, NLTK, spaCy, Lingpipe but AFAICT spaCy is competitive.
Regarding disambiguation: People usually mention books with their respective authors, so it takes one query against the local database to check whether entity is indeed a particular book by a particular author. When no author is mentioned, I check parent comment (if any) to find if the given book is mentioned there. Well, and if comment turns out to be standalone then I query Goodreads / Google Books API. Given books with identical titles, these APIs return the most popular option.
[1] Very rough prototype: https://confabulator.io/newsclustering/
a friend and I are working on a tech investment news aggregator (manually curated right now -- http://circulaat.com), and I think your clustering algorithm might be really useful for it. I can't figure out how to DM you via the HackerNews system but would love to get in contact
Detecting illegal mining activity via satellite
network intrusion (packet analysis)
Predicting machine failure
Stock market trend prediction (up/down)
Simbox fraud
Finding similar parts for large vehicles to help engineers
I can keep going, but this is a sample I've seen over the years. We mainly do work in time series data for enterprise basically mostly "things google doesn't do". A bulk of what we do doesn't appear in research papers because it's not "GAN art" eg: "the current hype".
* resource demand forecasting
* balancing distributed energy resource loads
* automating metadata generation
* categorizing 'events'
* finding anomalies in satellite data
Most important:
* running bots on my xbox
To see the results - post your resume here: https://www.postjobfree.com/post-resume
-Financials analysis / predictions
-Image classification