Toonify Yourself | Hacker News Reader

Toonify Yourself

toonify.justinpinkney.com

180 pointsdsr1293 comments

The examples on the landing page are not representative of the quality of the service.

Try it with internet photos of people. What it does is just morph the face a little: larger eyes, shorter forehead, elephant ears, narrow and taller mouth. The only thing it tries to generate is hair, and it fails horribly at it if you try with completely bald heads.

I decided to make this go full circle and grabbed a picture from thispersondoesnotexist.com, and toonified it. It worked fine.

I for one welcome the singularity and all it entails.

Interesting idea!

Good grief! I know I'm not Prince Charming but I was hoping for slightly less Shrek in mine. I think I'll be going back on that diet...

It shrekify's all men look at leo in the examples. I think it was mostly trained on women's faces. Those seem to get better results.

LOL. My wife said "you look like the dad from the Croods." It definitely seems to over caveman a bit, but I liked it!

I had the same experience. I'm not quite that much of a knuckle dragger.

https://imgur.com/a/4YqFPVl

You look like you walked straight out of Moanna

You're welcome!

> If you're seeing this message, that means JavaScript has been disabled on your browser, please enable JS to make Imgur work.

That surprised me, I didn't expect an image hosting website to require JS.

How else could they render the pixels? /s

https://i.imgur.com/2V5OlVz.jpg should work better.

The Toonify produces avatars which match the source photo in terms of detail and feature correspondence. In that regard, at first take, the algorithm might appear to be working at a near pixel level of granularity. However, at least in the examples I tried, there is a loss of similarity and recognisability, to the extent that I don't think my friends would get that my avatar is me, and I don't get that feeling about the avatars generated from pictures of my friends. For that reason, I am wondering if the algorithm is based on feature recognition (eyes, nose, mouth, eyebrows etc), followed by a feature level of replacement/distortion rather than pixel level?

The celebrity examples might be a different case in terms of our recognition sentiment, because of their familiarity, and the juxtaposition of a number of phenotypically diverse celebrities in one set, which gives a correspondence in the diversity of the input set and the output set.

What modifications would be required to the algorithm to score consistently higher in facial recognition of the avatar by people who view the avatar cold without access to the original photo?

I'm asking rhetorically, of course. Such research might take a long time. Toonify looks like a good foundation for what might be an open ended project.

This is explained in more detail here: https://www.justinpinkney.com/toonify-yourself/

Specifically, there's this snippet: "These StyleGAN face models can produce a huge diversity of faces and it’s actually possible to find basically any face inside the model. It’s actually a straight forward process to search for any image of a face in the model. So given an example image you want to find you can find a “code” (aka latent vector) which, when given as an input to the model, will produce an output which looks almost exactly like the face you’re looking for."

So it seems like they first generate an image that is similar to yours with one model, and then translate that too its more cartoony counterpart.

Ah! Gets excited.

1. Clicks on the browse button to find a good selfie.

2. Uploads the pic to Toonify.

3. Clicks on 'Toonify' button:

> Failed: Didn't find any faces in the supplied image.

Ack what? Ok. The machine is still learning.

(P.S: I like the project though. Hope it will improve over time. But toonification of any pic without deep learning/machine learning thrown into the mix is already a solved problem, no?)

Same here- looks incredible but no face found :(

If you were trying on iphone this should probably work now.

We're using dlib's frontal face detector for the face detection. It's ok, but certainly not state of the art, maybe in the future I'll find a better face detector...

The way I solved this for Faceweave is, if no face detected, let the operation proceed anyway. This gives the user the ability to line up the face themselves.

The logic is basically: ask dlib for face landmarks; if no landmarks, resize image to 1024x1024 and proceed; otherwise align face.

If you happen to like lisp: https://github.com/shawwn/stylegan-server/blob/10c396fa2ef07...

Also, congratulations on launching this! Super exciting.

Readers, if you use twitter, be sure to follow the creators:

Justin https://twitter.com/Buntworthy

Doron https://twitter.com/Norod78

Been following them for like twenty years now. They tweet a bunch of awesome stuff about ML, and stylegan in particular.

I tried my face with two different backgrounds but nothing. I guess I just have one of those faces! (might be a good thing)

At you on iPhone?

Rotate your phone 90° clockwise then try again.

By default the portrait image will be rotated, causing the face detection to fail. I know this because I‘ve run in to the same problem before, had to correct it server side :)

FAQ 1: My face is not detected.

Answer: Please check that your genetics is compatible with ML?

https://www.wired.com/story/best-algorithms-struggle-recogni...

"We don't store any of the images uploaded or generated. Once the toonification has finished running no images are kept on our systems."

Sure, then don't hide an otherwise perfectly normal computer program under a random web interface.

A perfectly-normal program that might only load/run on a Google Cloud TPU or a Titan V, mind you; and probably one held together with tape and bailing wire (hardcoded paths, "works on my machine", an undocumented and unreproducable build env, etc.)

Y'know, the same reason that the "perfectly normal computer programs" that generate the data for scientific journal papers aren't usually published.

StyleGAN of this size runs fine on your average CPU.

I know that for a fact since I wrote paint.net plugin that does exactly that. Said plugin works perfectly fine with images of the size they use (512x512) and generates output (CPU-only, mind you) in less than 5 seconds on a dual-core laptop...

I understand that some networks might require the compute power that you mention, but this just seems like an average CNN mapping images to Disney characters. And I don't have much experience on that front, but like qayxc said, this should run fine on consumer hardware.

And yeah, many journal papers suck specifically for the reason that their results are not reproducible.

Does someone remember "Kai's Power Tools"? The sample pictures instantly reminded me of that...

I remember reading reviews on this app (KPT Bryce [0]) in MacFormat in the 90s or so. The software seemed to use a really unique UI for development when compared to most creative Mac apps of that time.

---

[0]: https://en.wikipedia.org/wiki/Bryce_(software)

It seems to turn every male into an Indian.

LMAO

Presumably there is some kind of bias in the training set causing this. It does seem subtle but noticeable, although I wouldn't say it's just the males.

It doesn't appear to do well with glasses or baldness.

Just like real life

Or winking, or beards.

@OP, for images from iOS, portrait photos are uploaded in landscape mode, and you need to rotate them before the image pipeline.

Currently I believe that is what is causing your facial detection step to fail.

If you want to try this on iOS, rotate your phone clockwise 90 degrees then take a photo.

Yes, I'd just come to that conclusion too. (Thanks for the tip anyway). I've just pushed an update so this should work now.

The visual style reminds me somewhat of "Team America: World Police".

None of the `toonified` images shown on the page looks like the person in the photo.

OTOH I could immediately guess if you've shown me one of these photos: https://duckduckgo.com/?q=leonardo+di+caprio+toon&iax=images...

Yeah, I was hoping it was going to be trained on photo -> cartoon of same subject by some known cartoonist. Could be a fun profile photo that makes it look like x paper's cartoonist has thought you significant enough to warrant a cartoon.

Not sure what models you trained against, but my wife looks like a Kardashian. Think I can see Ray J behind her.

I look like a drunk Simpson... Barney.

Spot on!!

What gets demo'd on the developer's laptop: [Idris Elba]

What gets shipped to production: [Toonified Idris Elba]

https://assets.justinpinkney.com/toonify/images/vllkyt5r43p1...

Also available here, https://deepai.org/machine-learning-model/toonify

There are two plausible measures of success for something like this:

1) Whether the generated character is readily identifiable by an observer on its own.

2) Whether the generated character is a reasonable manifestation of the chosen art style.

I think it does a reasonable job of #2 and a mediocre job on average of #1. I think greater eyebrow variability would help some so that it doesn't look like all the women get "woman's pencil eyebrow" and all the men get "man's extremely bushy eyebrow".

I know HN is not normal, but I immediately tried my pets since I'm not happy to give out photos.

Didn't work, it can't find the face.

Otherwise it worked ok on the fascist leaders I tried. Black and white got color added. And one photo had an artefact put on top of both the original and the Toonify, a mirror of the top of the image to fill space perhaps?

Well done.

Also Startup idea, facial recognition for pets and sell the data to the Russians for addition tracking ability on people through proxies.

Welp, that'll replace the whale in me nightmares.

...wait what. I need to know more about these nightmares that involves whales. Did you stumble across a particularly vivid bit of whaler fan fiction that combined Melville with Lovecraft or something?

https://imgur.com/r/TheSimpsons/0F2Kvyu

This works really well when it comes to making scary pictures of my three year old niece.

I look forward to seeing if she recognises herself and what she thinks. I don't believe any adult likes a cartoon version of themselves, rarely are critiques truly flattering, a true cartoon hurts!

I tried cartoonifying my niece's favourite cartoon characters. The program failed. So you can't toonify a toon.

This is so cool! I'm curious, how much does it cost to run a server that does deep learning inference on the scale of hugged-by-hn-frontpage?

If people are still interested the site now back up and free to use!

> We don't store any of the images uploaded or generated. Once the toonification has finished running no images are kept on our systems.

I imagine this is all in good fun, but this disclaimer doesn't rule out data being derived from your face and stored or sold later.

I made this and can assure you I'm not doing anything with the images. This is just a silly side project of mine (along with Doron). I'm certainly not hoping to gain any money or data from it!

You should probably put this in the disclaimer. These days I assume all toys like this are data gathering tools in disguise. And I'm right more often than not.

He won't do that. then he cant sell the data later.

Could he do that though? I mean, hypothetically. I'm based in the EU so GDPR applies, and a picture of my face is definitely personally identifying info. So, unless I clicked something saying I consent to the data be stored/sold, I assume it would be illegal to do so.

Can you update the FAQ to stipulate that you aren't collecting face data too?

Because right now you say you're not doing anything with the images, that doesn't rule out collecting face data and storing that instead.

If you wanted faces for data, this would not be the way to do it. You could easily get 1000x photos of faces with a simple web scraper.

No words can describe the nightmare this just created out of my pictures. Hahahah wtf.

I tried it. But now I’m sure my photo will be used to forge passports.

I'll be really impressed when one of these that actually retains some of the original facial features instead of just replacing them entirely. Cool side project though.

Every image I tried got a "Failed: Didn't find any faces in the supplied image." Maybe it my images maybe the server has been slashdoted idk

I look awful! LOL guess i dont have a future as a toon. LOL

Not looking good here either, for once I’m happy with my original photo.

Oddly enough, I bet if the output was designed to always generate more attractive features (ala insta filters), this will take off.

Maybe I don’t high cheekbones or a chiseled jaw, but lie to me dammit!

Just a tip.

Super fun! Would it be possible to do the opposite? :)

Something like this?

https://thispersondoesnotexist.com/

I was more curious about reversing Toonify Yourself's input/output–upload a toon and get a real person's face back.

If you need some nightmare fuel, try running the output back through for another round of toonification four or five more times.

I was curious so I ran the Tim Allen on til it couldn't find a face anymore (4 times): https://imgur.com/d1FlNVh

Loved the results. Worked fine for me. Tried with a 28Kb JPEG image with lot of noise, but turned out good.

I know the answer is no, but anyway I could generate a full-blown character model from this ?

Surprisingly yes!

Or rather, the answer is "no" right now, but I will bet you $500 that it'll be "yes" within ... hm ... 2 years max. And if I lower my bet to $100, I'll do 1 year max.

Let's just say it's a very lucrative, very active area of research.

One observation: facial detection gives you landmarks, and those landmarks are essentially UVs for a texture. So if you do that process repeatedly, and rotate the face slightly using latent directions, then you can back-project the result onto a 2D texture.

EDIT: Some leads: https://twitter.com/Zergfriend/status/1283954193946906625

A video from July: https://www.youtube.com/watch?v=mvk3dtdeXBE&ab_channel=Compu...

And I saw someone use Unity to learn normal+bump+diffuse+ambient occlusion maps using ML, but sadly haven't been able to find the link yet.

This search gives lots of useful results: https://twitter.com/search?q=stylegan%20texture&src=typed_qu...

Very very cool. I could see this being intergrated into something like fortnight in the very near future.

If I was a better coder I'd being trying to get an Epic Grant for this. I can't imagine getting a full blown model would take more than a few hundred thousand

That was amusing. I sort of look like I should be an extra in "Braveheart"

Have you tried using an unpaired image-to-image translation algorithm, like CycleGANs?

Oh god, it has difficulty to deal with long beard! My result is terrifying!

It made me look rather a bit horrifying! But fun stuff, nice work!

It likes to point the eyes in different directions

Rate Exceeded :(

Beardy people end up looking..... scary.

I saw that too. I have a rather long beard, and my toonified photo was -- horrifying.

Feature request: webcam support.

Oh hey, more data gathering.

This shit is scary

glasses mess it up

this is cute, but to be honest, the celebrity ones are pretty bad - they don't really look like the person nor capture their spirit, with a few exceptions: Tim Allen and Gal Gadot are quite good and Will Smith is alright. Some especially bad ones: Leo DiCaprio, Barack Obama, Johnny Depp, Keanu Reeves, Idris Elba, Ryan Gosling, Robert DeNiro

Wow, you're really picky.

I agree, they don't "capture their spirit". No, you are not picky.

yeah the celeb ones are crap

I think Leo and Scarlett are pretty bad, but the other ones seem okay?

Oh, I didn't click "more." Still, I think most of them look reasonably okay. Pretty cool!

OMG if you toonify a toonified image it will give you nightmares this is a toonified toonified toon https://ibb.co/PczgpH1

Like an Escher painting.

Looks quite Modigliani.

Reply on news.ycombinator.com