Selfie 2 Waifu | Hacker News Reader

Selfie 2 Waifu

twitter.com

270 pointsguu94 comments

Twitter thread with examples:

https://twitter.com/tkasasagi/status/1250427941567094786

As usual for this sort of thing, the results vary all the way from “impressive” to “absolutely horrific.” I will say it seems to work quite well for many of the women in the thread.

It definitely prettifies most people although for one guy it took out his mouth. For the most part it “airbrushed them” and prettified the subjects.

It reminds me of the Twilight Zone episode "Eye of the Beholder"... maybe medical staff were just computers with some wonky ML for standards.

It can work for men OK too.

Bumped into this about a week ago and after quite a few tries it looked pretty good.

Figured to toss a few dollars to the person who created this via the paypal link at their site. But then paypal told me: "Currently PayPal accounts in China are only able to send payments. This recipient is not eligible to receive funds."

Meh..

Emailed the author to inform them and they mused about using bitcoin, but that the threshold is too high for casual users.

Perhaps time to reconsider.

Sigh, one more reason why we really need a strong paypal competitor.

The problem there is not PayPal. It is the company formerly named GoPay, a Chinese company, that necessarily adheres to Chinese law.

Otherwise known as cryptocurrency?

Send them cryptocurrency.

I tried it with a few photos. Best results with images with low-contrast lighting on the face.

I think the bad results occurred when there were too many variations in shadow and color on the faces.

People with clean lines and smooth gradients on their faces came out better.

I’m curious what the training set was for the model. I’m guessing perhaps a set of Japanese imageboard threads where artists draw manga interpretations of portraits. (These portraits tend to be of photogenic women, generally of east-Asian ethnicity, so it makes sense to see that most of the better examples in the thread seem to fall within that area.)

Yeah, I have a goatee, and ... my waifu ended up with no mouth or any other lower-face features.

Well that's terrifying:

https://twitter.com/jonathan1moore/status/125044976841907404...

Shadows seem to fool this thing.

It think it's more that it doesn't really get men.

yeah it literally says waifu

I wonder how much harder it would be to get the training set for selfie 2 hazubando.

Has worked fine on some of mine but they were beardless. I suspect because there's not many (if any) bearded characters in the training set.

Wow, this is way better with women’s faces. Facial hair in particular seems to trip it up.

Probably due to a training bias

The submitted URL was https://waifu.lofiu.com/index.html. Since the thread is filling up with complaints about redirects, spam, and malware, we've changed the URL for now to the one mentioned here: https://news.ycombinator.com/item?id=23032129. If the original site settles down, we can change it back.

I get redirected to a spam website from this page.

https://postimg.cc/svwnR4rK

So do I. Do you have any idea what it was meant to be?

You upload a picture of yourself, and it returns an anime-style version of that picture.

stylegan from selfie to anime

Me too. It seems that the site is compromised and issues redirects to malware without user interaction. Maybe @dang can consider changing this to an archived version of the site until it’s fixed?

It seems to be the ads, with an adblocker on in Safari it doesn't happen, turn it off and it sends me to a site encouraging me to install malware

same here.

The creator wrote on their FB page:

"Sooooo many people here. Thank you for visiting :) Gotta upgrade my servers. GPU server is the bottle-neck, but my credict card billing is flying Sigh ~"

And someone replied "put some ads on there", so the creator hastily slapped some ads on there to help with the server bills, and... this happened it seems.

Sorry for that, already get it clean.

> The selfie dataset contains 46,836 selfie images annotated with 36 different attributes. We only use photos of females as training data and test data. The size of the training dataset is 3400,and that of the test dataset is 100, with the image size of 256 x 256. The size of the training dataset is 3400,and that of the test dataset is 100, with the image size of 256 x 256. For the anime dataset, we havefirstly retrieved 69,926 animation character images from Anime-Planet1. Among those images,27,023 face images are extracted by using an anime-face detector2.

These seem tiny, don't NNs need more samples to achieve decent quality?

The CycleGAN datasets [1] all have less than 10k images. The two largest have 10,345 images and 5,129 images, the rest (like the famous horses2zebras) have less than 3k.

[1] https://github.com/junyanz/CycleGAN#datasets

One day people will have computers embedded into their optic and auditory nerves.

They will quite literally see a fantasy world, with filters and augmented reality changing everything they perceive.

Deep fakes and 3D transforms will mean any horny people can see anyone they want to naked. Other people will be able to live in a complete Lord of the Rings fantasy world with dragons, magic, elves and orcs.

Have you started writing yet or is this just teaser?

No, I honestly believe that this will happen one day.

This concept is very entertaining. The PS3 game Haze and Black Mirror episode Men Against Fire have similar premises. Are there any other works of media exploring this?

One of the more famous ones was the classic game (now, series) Syndicate. The core premise was exactly as described above - a chip that would get wired into the brain to alter all sensory perception. The idea was that the world in the setting had become awful enough that this level of personal surrender was no longer outrageous - people were primarily just altering the perception to be "nicer".

The core plot of the game then assumed that once a critical percentage of people had these, it was a viable tool for criminal organizations to engage in massive campaigns of thought control and overthrow the governments of countries all over the world, essentially replacing states entirely.

Illustrated in the Syndicate Wars intro: https://youtu.be/CrruSboN1bQ

Dennō Coil[1] is a wonderful and innocent take on this.

[1] https://en.wikipedia.org/wiki/Denn%C5%8D_Coil

Jack Vance's scifi/fantasy classic "The Eyes of the Overworld" starts with a quest which has the protagonist steal magical eye glasses which make the denizens of a village who literally live drenched in shit and mud and eat garbage believe they are lords and ladies living in palaces and eating the most delicious foods.

They know it's not the truth, but for all intents and purposes, while wearing the magical eyeglasses it becomes the truth.

The rest of the book is hilarious and I fully recommend it.

The book "The Golden Age" by John C Wright has this in a fairly major way (as in it's presence is an important part of the plot).

>Deep fakes and 3D transforms will mean any horny people can see anyone they want to naked.

A VR porn version of deepfakes... this is already close enough to possible that I would be shocked if someone weren't already working on it.

C.f. Ghost in the Shell, Rainbows End, and certainly many others.

...and all of that will be illegal hacks to the official form of the tech, which is supposed to show what the content pushers intended you to see.

The right to run ad-blockers, which would, in passing, enable content alteration like you suggest, would be something we'd actually have to fight for.

Most people wouldn't understand, and would be stuck in a world where they can no longer control what they see or hear, or how what they see or hear gets changed before their brain is allowed to perceive it.

There will be a person on the street corner crying out that the world isn't real... but only the people running hacks and filters will hear her.

I love those science fiction stories where everything is shiny and all the tech is amazing and just works. But, there's no future I can see that goes from here to there.

Already clean some inappropiate ads, sorry for that. Should be clean to visit now.

I am still getting tons of spanmy popups when I click "take a selfie".

Attention! This is a dangerous site. Don’t click it as it linked to numerous malware applications— at least 6 viruses there. For public interest.

The Github page has a lot more information: https://github.com/taki0112/UGATIT

Might be worth changing the link to this.

So, as someone with literally no experience with using this type of software, how do I go about self-hosting this? What are the hardware requirements?

The following instructions are for Linux or macOS. It may work on Windows too, but I'm not very experienced with Windows. No special hardware is required (you can run it on a CPU; no GPU needed).

Install pyenv[0] and pyenv-virtualenv[1]. Clone the repo and set up an environment:

  git clone https://github.com/taki0112/UGATIT
  cd UGATIT
  pyenv install 3.6.10  # [2]
  pyenv virtualenv 3.6.10 UGATIT
  pyenv local UGATIT
  pyenv activate
  pip install opencv-python==4.2.0.34 tensorflow==1.14.0

Download the pretrained weights from https://github.com/taki0112/UGATIT/issues/50#issuecomment-53.... Extract them:

  tar xf ugatit100.tar.xz
  mkdir checkpoint
  mv UGATIT_selfie2anime* checkpoint
  mkdir -p dataset/selfie2anime/{train,test}{A,B}

Crop your images in a 1:1 aspect ratio so that they contain only the head. Place them in the dataset/selfie2anime/testA/ directory. Run the program:

  python main.py --dataset selfie2anime --phase test

Open results/*/index.html in your browser to see the results.

[0] https://github.com/pyenv/pyenv#installation

[1] https://github.com/pyenv/pyenv-virtualenv#installation

[2] Other versions may work, but this is the version mentioned in https://github.com/taki0112/UGATIT#requirements

Thank you so much! I'll give this a crack on windows and see how it turns out.

I got it working via WSL on windows. Thanks!

I would be very careful with sending data here as the website has no Privacy Policy. We don't know how this data is being used.

This website however, present a Privacy Policy: https://selfie2anime.com/

I am not promoting one site or another. I did not read the PP of the selfie2anime.com either.

You can check the other implementations here (currently only two):

https://github.com/taki0112/UGATIT#web-page

> We don't know how this data is being used

what can possibly go wrong ? or just paranoid ?

Hmm, I just tried this with a few pictures, including one of mine. I forgot about all the "your selfies are being data-mined by the bad guys!" stories.

Ever heard of browser fingerprinting? If this site is doing it, now it has my IP, my browser fingerprint, and my face (as well as some friends' faces...). Search for my face on Facebook and they have my name too. Sell to marketer.

> Search for my face on Facebook and they have my name too. Sell to marketer.

Spoiler: Clearview already did this. And possibly others we haven't yet heard of because they've been smart.

The public sidewalk has no privacy policy, either, and I show my face there all the time. Same goes for the airport, restaurants, every ATM, and every convenience store that takes videos of my face.

Face imagery isn’t secret.

Every photo I upload keeps getting rotated the wrong way. Oh well.

If you're on your phone, take a screenshot of the photo to "bake" the rotation in

This is quite a useful tip. A lot of things can't handle the rotation metadata.

Let me guess, you're using some kind of Samsung Galaxy?

iphone 10s

The results: I wasn't impressed with the results, it seemed like I could have done better using some OpenCV and first principle algorithms. The examples they give are excellent, but when trying them they are awful.

The viruses: I didn't see all these viruses people were talking about, the software worked "as intended" for me without the redirects. There was no advertising on the page either... I run with uBlock Origin in Chromium.

The dataset: Where do these images come from? These seem like the sorts of pictures that might be uploaded to some older social media website, like Bebo or something.

2 cents: Personally I would like to try and do some work with a dataset that has a 1:1 mapping of person to character, but that would require artists and quite a bit of money. I was thinking that you could build out such a database for free by randomly matching anime characters and real pictures together and have a human rate closeness - then train a network based on these ratings.

> it seemed like I could have done better using some OpenCV and first principle algorithms

I am happy to volunteer a whole bunch of images (of me, sadly) as a test when you get this up and running.

Mine looks really good.

One wonders if sites like this exist to link faces to IP addresses, though. Get someone to upload a selfie from their computer, then you can add it to your database. Share on Facebook and Twitter, and they know your socials. Next time you walk into a store (post-mask era anyway), they find you in the database, and can see if you've visited their website or not, who your friends are, what sort of things you're interested in, etc.

I'm almost surprised that this isn't more common, now that I think about it.

Edit to add: I had to flag this link. The site is now redirecting to malware. I guess people did in fact want that database!

Not necessarily as nefarious, but just using sites like this to collect training data is probably a very common occurrence.

They link to their GitHub project. I hope to try it out locally this weekend.

I see the taki0112/UGATIT github project that it's based on but I don't see the project itself linked anywhere. I could easily be missing it though.

I couldn't try because it has some of the most aggressive ads i've ever seen. After I upload a pic it says my phone needs cleaning and sends me to a cleaning app on the play store. No time to check the results.

Somewhat related someone made this anime character generator I think it's pretty impressive make.girls.moe

Can be a time sink as you keep pressing generate over and over.

thiswaifudoesntexist is better me thinks

oh that's interesting and it produces a little story.

I don't know in the back of my mind if I ever really get into that personal assistant thing and want a wrapper/face around it, this would be interesting. Sadly many of the voices are bad... but there are those services you can sample a voice and get a voice to use for TTS, that could be something. Then licensing issues but yeah and latency from API trips

My waifu has a beard.

I'm not going to judge you for it.

My waifu merged my moustache with the shadow under my nose. That part is horrible; the rest is OK?

I'm tempted to edit my photo some, then resubmit it to see if it turns out better. It looks like it could be quite amusing.

Would love to get a chance to actually see the result, each time I scroll down to it on mobile I'm sent to a disgusting chain of ad redirects.

On iOS (FF or Safari) this site is a spammy nightmare.

Put some donation links (PayPal/bitcoin/patreon), try using teir 1 ad companies like Google AdSense and if they don't accept you, try something like native ads (outbrain)

I can understand how such GPU intensive website can be difficult to support for a creator.

Attention! This is a malware page— don’t click it. It directs to host of viruses, to me at least 6.

I gave both of them a whirl[0]. The second one is OK?

[0]: https://blog.ryjones.org/2020/04/30/ML

There seems to be a lot of flattening and smoothing going on, not unlike the art style of anime characters with no discernible nose.

A few casual observations:

* Looks like the expression is almost always the same.

* The backgrounds seem remarkably well replicated in many of these.

did you see the one with gandhi in the background? impressive.

I'm actually inclined to say that's less impressive because the model should be good at faces. On the other hand, the model doesn't seem to work when multiple people are in the picture. I'm wondering if somehow the gandhi face was recognized as not being a person in the picture, and was animeitized using a similar but different logic.

a good point.

It gave me scary results when I uploaded a few selfies that were black & white (grayscale)

I tried a few dozen selfies of myself with no luck. Maybe it's the beard? :(

results can be impressive if you look somewhat Asian

what's with the waifu thing, why not just call this Selfie2Anime/Manga or whatever.

It's an inside joke of the (western) anime community - cute anime girls == "waifus", as in you'd like to marry her.

https://www.urbandictionary.com/define.php?term=Waifu

I'm guessing the author wanted this to be aimed at people who get the joke and/or say that the project shouldn't be taken too seriously by using a meme in the title.

522 in Southern California

429 too many weebs

Proves "2d" is better than "3d".

My waifu was made by a dirty hippie on LSD.

Reply on news.ycombinator.com