Rendering Head in WebGL | Hacker News Reader

Rendering Head in WebGL

alteredqualia.com

238 pointsitamarb77 comments

Author of the demo here. Some background info about origin of the head model (of which my WebGL rendering is just a pale shadow, check following links for really high quality offline renders and also animation capture).

The head model was based on 3D scan using DIY scanning apparatus created by a single very talented person in his spare time - artist Lukáš Hajka:

http://pchiik.tumblr.com/

He documented his process here:

http://www.zbrushcentral.com/showthread.php?177615-DIY-Unive...

http://www.cgfeedback.com/cgfeedback/showthread.php?t=3776

At first it was just static model, later on he created a pipeline also for capturing animations:

http://www.youtube.com/watch?v=MgceFBMYvSU

http://www.youtube.com/watch?v=GOV7wvMZFBs

Lukáš deserves most of the praise, the model is just great. Shading I used here is actually pretty simple, I used more complex techniques elsewhere yet the overall results didn't look as good as this model with a simpler technique.

Here are some of my earlier attempts, each using a different technique:

http://alteredqualia.com/three/examples/webgl_materials_skin...

http://alteredqualia.com/three/examples/webgl_materials_bump...

http://alteredqualia.com/xg/examples/forward_materials_skin_...

You have several really great WebGL demos on your site, e.g. animation_physics_level

For some reasons they don't work in IE11 which has WebGL support. Is it because of Three.js or your xg library?

I try to push the limits of WebGL, I'm using almost everything there is, plenty of extensions (bear in mind these are not really exotic features, most are supported by majority of GPUs out there, native desktop games use much wider set of capabilities, it's just that WebGL implementations are only exposing them slowly).

In general Chrome has the most complete WebGL implementation, though Firefox does seem to be catching up recently (they do push for WebGL 2 so some features are popping up in WebGL 1 as extensions, but for example Firefox uses much older version of ANGLE and also no DX11 ANGLE in sight there, so on Windows Firefox is still quite behind Chrome).

IE11 WebGL on the other hand has troubles even just with basic WebGL specs, it still has broken and not implemented features, not even speaking about majority of extensions.

Is it possible to reproduce with something standard that most browsers can use, e.g. Unity3D or Flash?

You miss the point of a WebGL demo

Is the source of limitation browser implementation of OpenGL? Or is it a combo of the browser + OS? Or Browser + OS + graphics drivers?

The page header mentions "filmic tonemapping", I'd love to hear what that entailed in this case - is it a post-render 3D LUT as some games now do? I couldn't see anything from reading the source... I'm really interested in how we perceive quality of colour in CG renders, and how CG in movies which goes through a print film LUT is helped or harmed by the process.

I do tonemapping in a final compositing step of deferred rendering. Here are formulas:

http://filmicgames.com/archives/75

John Hable did a great job proselytizing virtues of linear space lighting and proper tonemapping.

http://filmicgames.com/archives/category/tonemapping

http://filmicgames.com/archives/category/gamma

There have been also SIGGRAPH course if you want to delve-in deeper:

http://renderwonk.com/publications/s2010-color-course/

Those SIGGRAPH talks, OCIO and Jeremy Selan's http://cinematiccolor.com have been a touchstone a while :) Thank you so much for the filmicgames link, that's exactly what I was curious about.

In return maybe I can offer the Academy's linear-to-display reference rendering transform, which is intended to replace the myriad of tonemapping LUTs currently used for film: https://github.com/ampas/aces-dev/blob/master/transforms/ctl...

The glowing eyes before the skin had loaded was a bit scary, but overall, good work. However, is it possible to easily disable the heavy depth of field? I feel like that might be exaggerating the quality in a way that wouldn't be realistic for practical use, especially in terms of the hair/stubble over their skin.

I had a look at the "renderer" object in Chrome's developer tools. I tried renderer.dofEnabled = false but that didn't seem to have any effect.

Open JavaScript console, write:

renderer.setDOF(false);

I see a blue outline on the left of the head and a red outline on the right when I do that. I'm guessing that's some edge case for your lighting model?

This is actually on purpose, I'm trying to emulate a physical lens distortion, adding a subtle chromatic aberration:

http://en.wikipedia.org/wiki/Chromatic_aberration

You know, some time ago, I half-jokingly put forth the idea to some friends, that chromatic aberration will become the new lens flare, i.e. an effect to make rendered 3D graphics look like a movie shot with a cheap camera by an inexperienced cameraman on purpose. I get really sad each time I see one of these effects - they make the graphics worse on purpose to make them look like a bad movie and it wastes processing power.

If you do want to simulate a camera lens, I'm not going to stop you and wish you luck. However, if you want to render 3D images for humans to look at, I'd suggest putting in effects that are observable with a naked eye and not through a glass lens. Stuff like bright light blinding you, eye adjustment to varying light intensity and motion blur. I realise it's a bit harder to implement these effects, since you can't take a reference photo and compare, but I would really like to see someone try to render images as seen by a human and not a camera.

Thanks! With it off, I do notice that whatever is making the stubble seem like it protrudes(bump? normal? parallax? Been a long time since I've even thought about this stuff) seems a bit low resolution compared to the skin texture itself, as the light rises with the hair in every single case.

Still though, I'm genuinely surprised at just how well it held up without the DOF. I would've thought it'd be much worse, but frankly it looks great. Great demo.

Yup, bump map is indeed lower resolution (1024x1024) than diffuse skin texture (2048x2048).

It would be better to use higher resolution one, but these were the data I had at hand.

I also made a quick-and-dirty 2K gloss map so it should help a bit with adding surface details even if bump map is lower res.

Gloss variations give surprising amount of surface information - in another demo at one point of time I had bugged bump mapping and it took me a while to realize it was broken because gloss map was working so well.

There's quite a few good WebGL demonstrations to choose from on the home page [1].

[1] http://alteredqualia.com/

Thanks (creator here). Please be gentle on my server, right now it's rather overloaded ;). This particular demo is quite lightweight (2.5 MB), but some others are not.

It's all just static files, so they should load eventually, just it needs a bit of patience (I'm being pummeled right now both from here and Twitter).

If it's static files I'd recommend the cloudflare free tier to off load them on to their CDN. https://www.cloudflare.com/plans I'm not affiliated other than being a customer...

Anyone else running into blurring issues? I tested it out on Firefox and Chrome, and in both browsers the face texture is a blur. The high resolution only appears when I have the browser scaled horizontally small enough to cut off half of the head. If I scale it larger, it blurs out again.

http://i.imgur.com/Yd4jCme.jpg

That's probably broken autofocus. It's done fully on GPU so it may be a bit tricky on some not-up-to-date drivers I guess.

Which browser / OS / GPU did you use?

Does this demo focus ok for you?

http://alteredqualia.com/xg/examples/animation_physics_ammo....

Yep, that demo works fine.

Firefox/Chrome, Windows 8.1, Intel HD 4000.

Did you get any errors/warnings in the console?

  XG [version 0 | build 410 | 2014-03-30]  xg.min.head.js:20
  XG: render target support autodetection START (please ignore following warnings)  xg.min.head.js:12
  WebGL: checkFramebufferStatus: the internalformat of the attached texture is not color-renderable  deferred_skin.html:1
  XG: render target support autodetection END  xg.min.head.js:12
  [.WebGLRenderingContext]GL ERROR :GL_INVALID_FRAMEBUFFER_OPERATION : glDrawElements: framebuffer incomplete (clear)  deferred_skin.html:1

So far this was the only unexpected log.

Messages in other ones come regularly in capabilities autodetection (the only way how to find out rendering into which render target formats is supported is to do trial frame buffer creation and check for status, which also triggers built-in warnings).

But even this one could be bogus, I already noticed out-of-order logging into console before, so even this "framebuffer incomplete" error can be still part of the autodetection.

At this point I do not know, something is not behaving as it should be on these Intel GPUs, though curiously it does work ok on my older Intel HD 3000 under Windows 7.

If you can (on Windows), you can try to enable DX11 ANGLE rendering backend in Chrome and see it it makes any difference (more capabilities are available with DX11, I use different code paths when more advanced features are present).

It's available via chrome://flags/ under "Enable D3D11" and should work in current stable Chrome 33 (though in Chrome Canary 35 it does work better).

HD4000 here too. Here's what's in the console:

XG [version 0 | build 410 | 2014-03-30] xg.min.head.js:20 XG: multiple render targets not supported. xg.min.head.js:12 XG: fragment depth setting not supported. xg.min.head.js:12 XG: render target support autodetection START (please ignore following warnings) xg.min.head.js:12 3 WebGL: checkFramebufferStatus: the internalformat of the attached texture is not color-renderable deferred_skin.html:1 XG: render target support autodetection END xg.min.head.js:12 XG.DeferredRenderer: can't use multiple render targets, falling back to multipass.

And it looks like this: http://imgur.com/YjoGQPv

I'm getting a dark blurry version. The console has:

    TypeError: asm.js type error: expecting c.y where c is either the global or foreign parameter xg.min.head.js:2
    "XG [version 0 | build 410 | 2014-03-30]" xg.min.head.js:20
    "XG: half float textures not supported." xg.min.head.js:12
    "XG: linear filtering for half float textures not supported." xg.min.head.js:12
    "XG: fragment depth setting not supported." xg.min.head.js:12
    Error: WebGL: A texture is going to be rendered as if it were black, as per the OpenGL ES 2.0.24 spec section 3.8.2, because it is a 2D texture, with a minification filter requiring a mipmap, and is not mipmap complete (as defined in section 3.7.10). xg.min.head.js:11
    Error: WebGL: A texture is going to be rendered as if it were black, as per the OpenGL ES 2.0.24 spec section 3.8.2, because it is a 2D texture, with a minification filter requiring a mipmap, and is not mipmap complete (as defined in section 3.7.10).

I'm having the exact same issue on Firefox / Windows 8.1 on an Intel Iris 5100, so it's probably an issue with Intel drivers.

Also Intel here (on Linux). I guess you may be right.

I tried to boot with my Intel HD 3000 enabled (secondary GPU in my Thinkpad) and while it was very slow, demo did work.

I'm on Windows 7. Weird it doesn't work on newer GPUs and newer OS, regresses do happen sometimes.

It does work for me elsewhere, on my Nvidia Quadro 2000M and oldie ATI Radeon HD3650, Chrome and Firefox, with DX9 ANGLE, DX11 ANGLE and OpenGL rendering backends.

Having issues on ChromeOS too, can see the head and eyes but it's all very blurry.

Yes same issue here (Intel HD 4400), turning off DOF fix it:

renderer.setDOF(false);

I got a extremely blur rendering and I tried what bd suggested in another thread -

  "Open JavaScript console, write:
  renderer.setDOF(false);"

It looked fine after that.

Amazing realism. I just had a vision of meeting a character like that in a game - up close - wearing my Oculus (or better...)

And it was awesome.

Lots of exciting stuff ahead folks. This is going to get good.

I really like the way the limited depth-of-field makes it feel like he's really close to you. Creepy as heck.

By the way, I think a less disturbing title would be "Rendering A Head in WebGL" :)

Indeed, the first time I looked at the title I thought a "rendering head" was some piece of software (a front-end?) for rendering.

Not that "Rendering A Head" is any less disturbing considering the alternate meaning ( http://en.wikipedia.org/wiki/Rendering_(animals) )...

This skin layer image is so creepy...

http://alteredqualia.com/xg/examples/models/ctm/head/diffuse...

I'm going to bookmark this, in case I ever feel the need explain the Uncanny Valley to others through example.

Vaguely interesting: how much his emotions seem to change only based on his head/eye angle: http://imgur.com/a/6QEIq

Humans actually can't do this:

http://i.imgur.com/Ul3ti69.jpg

As we look down, the upper eyelid automatically follows.

Try it in front of a mirror. Tilt you head back and try to open your eyes as shown in the image above. You won't be able to do it.

Interesting, never thought of that.

Did you mean 'can't' or 'don't'. I just tried it, it's easy, it just feels like you're a bit wide-eyed.

slight tangent : can anyone recommend resources or entry level project ideas for anyone getting started with rendering APIs (OpenGL, WebGL, d3d, etc.)?

I'm not opposed to any of the JS wrappers (three.js), but would like to develop some intuition about the APIs that more closely interact with the GPU.

http://open.gl

http://learningwebgl.com/blog/

http://www.webglacademy.com/

is great for learning basics of WebGL or shaders alone.

Wow. Creepy. I think we found the NSA mascot. "We're watching."

Ok, that's terrifying. It needs to blink and should bee smiling.

I understand that it's a lot easier to acquire a static mesh than one that's rigged for animation.

Incredible result. Eyes are not on par with the rest, however. Could you add SSS material on the eye-white, I think that would bump up the realism a notch.

Good idea, added, thanks. It helped a bit, but eyes are indeed the weakest point.

Even if material rendering would be perfect (which it isn't), there would be still missing eyelids deformations and there are no eyelashes.

One day, step-by-step, right now I'm glad there are some eyes at all, previous model I experimented with had eyes closed ;).

> there are no eyelashes

So that's what was making me try to figure out why there was something not quite right about the face! It seems we tend to notice little details like this more unconsciously; it's probably a huge contributor to the uncanny valley effect.

Yup, it definitely looks a bit better now. The eyes could also have reflection maps, but they should be properly shadowed or it looks weird. Eyes are hard, I've not seen anyone do them perfectly yet, even in offline renders.

Or this: http://www.vill.ee/eye/

The initial black mask adds to the overall creepy vibe of this unsuspecting giant face presented before me. I wasn't going to sleep anyway.

Amazing! And it's 15fps in my Macbook Air with Chrome.

Setting DOF to false bump fps up to almost 25.

Wow that's so awesome! I'd love to see more things in that level of detail!

Pretty cool/creepy how the eyes keep looking at you when the head rotates.

I'm getting really fed up of WebGL not working on iOS!

I wish the author wrote an explanation on how it works!

Wow, what a difference between Chrome and Firefox.

Really? For me it looks exactly the same in Chrome and Firefox. Both seem to work (fotorealistic head with tiny details, nice lighting, depth of field). My FF version is 27.0, Chromium is 32.0.1700.107, OS is Archlinux, graphics card is from Nvidia.

What difference have you got between the two browsers?

EDIT: I found one difference, Chrome has 60FPS, FF only 37, at same resolution.

OK, a shadow of a head. Do I miss something?

Does it say "Loading ..." in the top-right corner?

In the beginning. Then it's gone and I have a black head in my browser. A shadow.

JavaScript allowed. Firefox 29 Beta. Mac OS X 10.9.

I guess it's a Chrome only demo again. :-(

Works fine in FireFox 28 and Safari 7.0.2 (both the current versions of these browsers) on Mac OS X 10.9 here.

The black looks like it is a bandwidth issue. It looked like a black head in FireFox for a few seconds, but then (I guess when additional data was downloaded) it looked similar to what Safari showed. FireFox is a bit darker and seems a tiny tad slower.

I get the same result here: also Firefox 29 Beta, Mac OS X 10.9. Maybe it's a Firefox Beta issue?

Here's what I'm seeing: http://i.imgur.com/dXLigH3.png

Ok, it's starting to get creepy :P

What is this? Is it just a high res texture mapped polygon or is it some kind of ray tracing?

Click link. Listen to video card fan speed up dramatically.

Could you try using Megan Fox's head next time?

Thanks.

That's too fucken creepy!

That's amazing.

Fun With Your New Head

Heads are so funny, and there are a thousand laughs in store for you in the new, improved HEAD. Everyone enjoys a talking HEAD, from young to old. Taste, see, smell, and "pain" with a HEAD. Experience every emotion known to the HEAD. And if you already have a HEAD, remember what the HEADS say: "Two HEADS are better than one!"

Everyone enjoys a talking HEAD. Every minute is different from the next minute in incredible thought-chaos of a HEAD. And every single HEAD is different!

HEADS are so funny. Listen to the limbless talking HEAD talk about "Freedom," "Death," "Beauty," and "God-Father." Make the HEAD fall in "Love" with you. Any HEAD can be made to "Love," if training manual instructions are carefully observed. Watch the worn-out HEAD die, talking, talking, talking till the moment in decays. Indeed, it is not an exaggeration to say HEADS are so funny.

Taste, see, smell, and "pain" with a HEAD. Every HEAD purchaser receives absolutely free a "Life-time" supply of "Food." Put "Food" in the HEAD's Mouth, then insert consensual Apparat into Left Collarbone. You will taste every molecule of the Mouth's "Food." Only those who have "eaten" with a Mouth can understand the incredible sensations of "Food."

Left Collarbone is likewise Input/Output source for Right Eye and Left Eye. See the strange little world of the Right Eye, looking at you! See through the Left Eye too! Then see through both the Right Eye and th eLeft Eye together. Every Exo-Export HEAD has two eyes. Don't accept less!

Left Collarbone is likewise Input/Output source for Nose. Now, with the new, improved HEAD you can experience the disconcerting primeval world of "Sex," as the center of the new HEAD's sex-tropic response is removed from obsolete and unsightly sacral area and redirected to the graceful Nose. Just one more reason why two HEADS are better than one!

Left Collarbone is likewise Input/Output source for "pain"-sensitive Chin. Throughout the galaxies there are creatures, often the most insignificant, that can experience the famous "Negative Pleasure," and now with a HEAD you can too! The new, improved HEAD is thirty percent more sensitive to "pain," thanks to refinements in the Chin.

Left Collarbone is likewise Input/Output source and control center for Adam's Apple. Nothing is easier than to take over your HEAD's talking- function. Amuse your friends by talking through your own HEAD! What could be funnier than to talk to another HEAD that thinks you are just a HEAD too?

Everyone enjoys a talking HEAD, from young to old. Even more fun than talking-function of the HEAD is thought-function. Insert compassional Apparat into Right Collarbone, and experience every emotion known to the HEAD. You will feel the HEAD's amazing "Love." You will be paralyzed with the HEAD's consuming "Fear" of pain and of its own inescapable death. You will hate your own self -- perhaps the most exciting sensation of all.

HEADS are educational. Everyone should have his own HEAD to grow up with. HEADS provide an easy and stimulating introduction to basic concepts of xeno-language and xeno-culture. Each HEAD is given a thorough grounding in the astonishing cultural traditions of its autochthonous planet. A third of a lifetime is devoted to the education of every Exo-Export HEAD.

HEADS are perfectly safe for the young. The sharp, skeletal teeth are extracted from each HEAD's Mouth at the time of assembly and refitted with harmless, hydraulic pseudo-teeth.

Many designers consider HEADS to be an attractive addition to the decor of one's environment, especially in arrangements with contrasting xeno-flora and xeno-fauna. For the fashion-conscious HEADS are available now in a range of natural tincts from brown through pink. When treated with new, special-formula Fungi-X, HEADS can also be cultivated in more agreeable colors, though fungifying processes will abbreviate markedly the lifetime of the HEADS so treated.

Everyone should have his own HEAD, and now everyone can! Thanks to the diminished Chest volume of the new, improved HEAD, the result of recent advances in biominiaturization, HEADS are cheaper than every before. They eat less and take up less space too! So why don't you buy your new HEAD today?

Any HEAD you buy from Exo-Export is guaranteed to be the native handicraft of its autochthonous planet, where bioengineering has long been practiced by the wild-four-limbed progrnitors and manufacturers of the HEADS.

There are a thousand laughs in store for you in the new, improved HEAD. Why don't you buy your new HEAD today? Why don't you buy your new HEAD today? Why don't you buy your new HEAD today?

Only 49.95 from Exo-Export Monopolies.

https://www.youtube.com/watch?v=ZqefPy3q3Do

And if you order before midnight tonight, we will include a free sample of HEAD ON: Apply directly to the forehead! https://www.youtube.com/watch?v=f_SwD7RveNE

The first step toward Oculus and Facebook integration...

Reply on news.ycombinator.com