Bringing the Mona Lisa Effect to Life with Tensorflow.js | Hacker News Reader

Bringing the Mona Lisa Effect to Life with Tensorflow.js

blog.tensorflow.org

212 pointsheadalgorithm27 comments

Really neat. That's some really great work there.

Maybe an off-topic comment, but as an older developer I'm increasingly feeling left-behind by newer advances, especially AI/ML. I remember the feeling of being a student and junior dev and feeling that a lot of code and frameworks and such was "magic." When I got more experience/education it stopped being magic and started being brilliant. I'm almost back to where everything new and cool is "magic" again.

You never need to stop learning. I can say with some confidence that the author of this article had not used TensorFlow.js 12 months ago.

Yes but one needs some time away from the computer especially when they have a family, eg. young children to spend time with. I personally accept that I’ll never catch up with all the new technologies but hope get to toy with the most significant parts in the future. At the same time I want to give the offline life a chance, there are old technologies such as acoustic instruments and phisical medium painting that are rewarding as the new stuff and they offer a different perspective into processes, sometimes deliberately slow and deep

You can if you can find just 10 minutes. See my reply above.

You think other people don’t have families or things to do away from the computer?

Woe is you

Ive been doing this technology thing for a long time, at some point other things become more important and take a higher priority. Im still holding onto a full time programming job but in my free time I do other things. If you’re young and excited about all this, good for you and enjoy the ride. At some point you might change your priorities if other things become more important. Cheers

I personally still love programming and perhaps when my kids are grown I’ll do it more in my spare time again.

I agree though. I don’t have a choice but to cram my learning into work. Thankfully I work with an awesome team and we love to experiment, so I learn pretty much weekly if not daily. The learnings are small, but arguably the important parts of programming are in those details. I’m content for now.

If I tried to set aside time for programming in my spare time, I’d have to sacrifice 1) time with family 2) time to exercise 3) time to take care of myself away from a screen 3) time to shut off. I want more of all of those, not less.

I do have friends without kids who program 10+ hours most days. They love it. I just need another 24 hours in each day, I guess.

> I do have friends without kids who program 10+ hours most days. They love it. I just need another 24 hours in each day, I guess.

I agree that once one has a family priorities change. Regarding the friends who program 10+ hours most days who don't have a family to go to, I think I was just like this a while ago. But thinking back I really think that is not a sustainable thing, especially if coupled with bad habits such as eating whatever comes your way and ignoring any physical exercise the body needs. It really does not look too healthy to me and wish I realized this a bit sooner than I did.

Likewise, I’m glad to have new priorities. I think what I miss is the progress and freedom to pick what I learn. These days I have to fit in what I can as I’m able to, which isn’t quite as engaging or exciting. At the same time, it’s a lot more sustainable.

Anyone who reads this can recognize any object in their room right now in less than 5 minutes of their time if they want to using TensorFlow.js model creation in the browser. ML is not so hard to get started with these days. Check this short tutorial on how to make a custom object classification model in minutes for a quick prototype if you are up for the challenge (5 mins of your time): https://www.youtube.com/watch?v=z05bjEEgVQc&feature=youtu.be...

That's how I felt until I watched a few videos about the low level details of neural networks. It's just basic statistics and calculus! Not just in a "it's turtles all the way down" kind of way, but in a "18th century king could have hired Leibniz and a thousand monks to run training on papyrus" kind of way.

Relative to impact, no other field of CS has implementation details as underwhelming as deep learning IMO. The magic is in the GPU and the high level libraries that let you make entire models with a few lines of Python.

Here's a 4-part video series by 3blue1brown : https://www.youtube.com/watch?v=aircAruvnKk Explains the big picture in an intuitive way

I was feeling the same way with programming flash loans and yield farming during spring, and mostly this summer, but I got up to speed!

Its been fun programming and deploying smart contracts! It feels like I'm not keeping up, but then I realize it hasn’t really been that long and it’s never been so easy to directly convert productivity into convertible money, over the internet

Kind of reminds me of "cookie clicker" / incremental games except in real life

haha brrrr

This is awesome! Great work on getting this running in tensorflow.js !

Question on business / strategy for my "startup" (actually just a side project) in this space.

I was just working on a server side streaming implementation of first-order-model for https://vo.codes

I'm trying to build a 1-stop shop for deep fake memes, and first-order-model and wav2lip are two of the video processing facilities I want to add.

My product pipeline is to add the video tools, a Twitter/Discord bot, and voice conversion (which I have working). I'm hoping to draw enough traffic that I can run a Kickstarter to give me runway to build a client-side real time voice and video conversion app for gamers, Twitch streamers, Discord, etc.

I already have real time voice conversion working, I'm just struggling with the time to do it as this is still in the "side project" phase.

Does anybody think this is a sound strategy? Does Kickstarter work well for product funding? I've demonstrated lots of interest (millions of uses of vo.codes, and growing), but my marketing strategy is still weak. How can I fix this?

I first saw vo.codes in the huge pandemic project thread and have been following it since then. Very exciting to see your user base and the Discord community around it grow.

Regarding monetization, have you considered either having a paid API tier or simply licensing the models for use? Anecdotally it seems like there are many hobbyists and companies messing around with synthetic voice projects and providing all the vo.codes voices could be a big value add for many of them.

Email us :)

Ive been rewatching all the Harry Potter movies recently, and have kept a very small list of cool things I’d like to try to replicate in the HP universe with software. So it’s funny to see this in the front page of HN, as it’s a perfect example of doing just that! I’d love to see some classic Renaissance paintings brought to life to display in the house, like the paintings which line Hogwarts.

Samsung's pricey but interesting "The Frame" line of TVs is designed to blend in as wall art when not in active use, including what's supposed to be brightness adjustment to match room light levels (though I haven't seen one in person to tell how much it works). Hook something like that up and you could probably have some fairly convincing 'living art'.

I'm still stunned we're able to achieve all this in a web browser today (not the processing power but how accessible this is to people). Great job, Emily!

Check out #MadeWithTFJS on Twitter/LinkedIn to see what others have made for even more inspiration!

The code can be found here: https://github.com/emilyxxie/mona_lisa_eyes

cool. reminds me of this simpler project which my kids had fun playing around with: https://github.com/yemount/pose-animator

Shan's pose animator is totally awesome. New updates coming ;-)

Tensorflow.js is awesome - I trained up a model for syncing 2d avatar lips to voice in pytorch and was shocked to find a top-notch LSTM inference implementation in tfjs that also uses WASM + SIMD to run extremely fast inference. (SIMD is coming to wasm in the next version of chrome.) It's some cool tech, I expect to see a lot of amazing stuff on the web from it.

I've been playing a bit lately with wav2lip (coincidentally, sometimes using a similar picture of the Mona Lisa). Even though it's a different use case, I'm definitely going to try some of those clever techniques that make the lower res generated video look less "copy-pasted"

This would be a really great physical product, although exceedingly silly.

The blinking is a nice touch.

Reply on news.ycombinator.com