MIT scientist captures 90,000 hours of video of his son's first words, graphs it | Hacker News Reader

MIT scientist captures 90,000 hours of video of his son's first words, graphs it

fastcompany.com

286 pointscanistr30 comments

"applies these same high-powered analytics to relate [...] events broadcast on TV to conversations taking place in social media"

So, if you use Twitter, you're someone's lab rat. I know that when I tweet, it's open for all the world to see, but I can't shake the feeling that this is really creepy.

Beyond that, I'm trying to envision how this data could be used. I imagine three scenarios:

1. Leverage it to sell more TV advertising. If a show is getting buzz on twitter, maybe this could be a viable alternative to Nielsen ratings. (Which leads to the question of whether Blue Fin can discern positive buzz from negative buzz).

2. If step 1 is analysis, could step 2 be manipulation? Can buzz be jump started or boosted at key inflection points by sock puppet marketing?

3. C-level eye candy - interesting visualizations that TV exec's buy for their egos, but ultimately can't do anything actionable with the information.

Unless they're sandbagging the spam filtering right now, there's no way twitter is close to tapping into the potentially huge value of their data. I'd love to see them turn the data into huge value and insight, but they'll need to show a lot more before this outcome is likely.

I feel like either the data Twitter has isn't actually that valuable, or Twitter is just sitting on a goldmine and they are refusing to show their hand too soon. I say this because they're still not profitable, but I predict that if they do become a large financial success, it won't be through existing ad-revenue models (or maybe even ads). Twitter could monetize in countless ways with their powerful information base.

I think that it will be used in more sinister ways. They will provide information on how to make political speeches and propaganda more influential and how to get main stream media stories to be better believed.

There has already been a statement on the particular psychological tricks Obama uses in his speeches - I can only see this being used to strengthen that.

Haven't politicians, leaders, and many others in positions of authority been doing this for millennia? People have been studying this (rhetoric) since ancient Greece.

Engagement is meaningless without knowing the sentiment behind that engagement. It's easy to see how many people are talking about your product (and it's not just TV producers who care about their product in this way); what is harder is to find out if they like it or not.

You present it as creepy, but (as long as we are only talking about public data like Twitter) I welcome it. The better the analysis, the better product developers can do at creating products I actually want. The danger for product developers is always building products that are just slight tweaks of the previous product (see the movie industry for example).

> So, if you use Twitter, you're someone's lab rat.

Among others, Noah Smith's (and his grad students'):

http://www.ark.cs.cmu.edu/

Here's the TED talk: http://www.ted.com/talks/lang/en/deb_roy_the_birth_of_a_word...

So the language acquisition stuff could be an n=1 just-so story, but meanwhile:

Deb Roy and Phil Decamp just invented a way to efficiently browse days of multiple video streams - on paper, if you want:

http://video.fastcompany.com/plugins/player.swf?v=2014e22b27...

Their work on "time worms" is astounding; unfortunately, the video ends before zooming out to the next step: multitrack timeworms with hundreds of cameras; broadband recordings of ALL THE CHANNELS.

http://www.youtube.com/watch?v=RE4ce4mexrU&t=11m56s

We can put off the hard problem of CV-driven video search; we can solve the much easier problem of "leverage people's brains to recognize patterns". Visualizations are the new algorithm.

I think a better summary would by "MIT professor collects 90,000 hours of video of his son's first words, gets tenure based on that, does nothing with the data. With tenure safe in hand, leaves his students in a lurch as he heads off to do a start-up."

It's a great corpus he collected. Wish he either had a longer attention span, or at least bothered to share the data.

I had a similar reaction. I think it speaks to the time, that some of the greatest minds of our generation are 'seizing the opportunity' of a startup to develop social analytics around the 'jersey shore'. I can't help to think the great linguistic scholars (ex. his fellow MIT faculty member, Chomsky) doing the same things in the 50's. Can you image the loss to humanity if each one, after giving a TED talk, abondons his research in their 30's or 40's to make an advertising platform? This made me sad.

I think it's not that bad he didn't use the data very much himself, as the research could probably not be considered very objective. Didn't he mention he shared the data at least with his research group? Hopefully they'll develop some method to preserve this family's privacy and share the data further.

On the last part, there's a slow move towards journals mandating that authors share their data, so that other researchers can replicate/extend/revise results: http://oad.simmons.edu/oadwiki/Journal_open-data_policies

This case is a bit tricky due to privacy concerns, though; it contains raw audio/video of years of his personal life, which even most open-data mandates wouldn't require sharing, at least not in original form.

I keep a git repository[1] of my son's (1,5 years old) keyboard hammering. He "types" until he accidentally leaves insert mode in Vim or leaves the computer, and then I commit it.

In 5-10 years time I hope to be able to do some real-time visualization with milestones like "first real word", "first use of complicated word", etc. If he's interested I might track his first code there as well.

1: https://github.com/jacobrask/Ivar

Out of curiosity, why did you choose to give a kid a modal editor?

I didn't really choose, I was busy coding one day and his mom had to do something quick, so I had him in my lap. He wanted to type like I did, so I just opened a split in Vim and let him type a few characters, he could barely press down a key anyway. When he later wanted to go to the computer even when I wasn't sitting there, I just continued in Vim.

I'll probably switch to something else now that's he's actually starting to understand the connection between the monitor and the keyboard better. Previously he mostly liked right-clicking with the mouse as it gave a big box on the screen..

it's a sensible choice to start him with Vi and leave Emacs for when he is a man~

I'd be afraid that letting a kid hammer away with a modal editor might accidentally reformat the hard drive or something.... But seriously, wouldn't you get better results with a plain, non-modal editor?

Any interesting results so far? Any interest in publishing his ``commits''?

It's on GitHub, see link above.

The biggest milestone so far was probably when he realized that the cursor moving on the screen was connected to him doing stuff with the keyboard. It was pretty fascinating to see, him looking at the keyboard, at the monitor, at the keyboard, then "touch typing" while looking at the monitor :)

Very cool good luck.

The MIT project sounds awesome and I love the visualization. I was underwhelmed by the offshoot company called Bluefin Labs. For such awesome research at MIT, it seems like Bluefin is basically Twitter trending for TV shows.

I was disappointed.

The graphics/visualization is breathtaking but what they learned from all of that analysis seems very limited: Apparently, when caregivers discover that a child is learning a word they simplify their use of that word till the child grasps it and then revert back to normal usage once the child can use the word.

It could very well be that their regular usage of the word that the child is learning is unaffected, but in addition they use the word more often in simple sentences around the child, and once the child understand the word revert back to regular usage patterns/frequency. That would explain the results and does not seem that interesting.

I am studying Natural Language Processing, so I have been interested in how humans learn language, in order to understand if the process can be replicated such that a computer can learn a human language. I think we would have to read the actual paper to really understand their findings, but the main thing I took away from this is: in order for a child to learn a language the parents apparently subconsciously alter the way they speak (whether it is just to or also around the child), language learning appears to be tied to psychical locations and children seem to rely heavy on feedback by their care takers.

That would be an indication that there is a limit to what computers can learn about human language by analyzing textual data (like e.g. Google Translate does). This is related to the poverty of the stimulus argument in linguistics. (http://en.wikipedia.org/wiki/Poverty_of_the_stimulus)

If only teachers were taught to do the same at school.

</troll>

It might be more interesting if it was a new discovery, but it's known as infant-directed speech, and has been well researched.

This is probably one of the coolest projects. The people working on this project are doing and figuring out some amazing things.

Has the poor child ever got outside the house?

Not related to his project, but I feel very proud in a distant way. My last startup's office (in Kolkata, India) was at his father's house. Don't know why I wrote that, just felt...

I liked the example with the president's speech. It won't be long before politicians will adjust their speeches during the speech according to the reactions from viewers.

...then public dies of ennui.

Reply on news.ycombinator.com