lcrs

There are a few projects now that simulate defocus properly to match what bigger (non-phone camera) lenses do - I hope to get back to working on it this summer but you can see some examples here: https://x.com/dearlensform

Those methods come from the world of non-realtime CG rendering though - running truly accurate simulations with the aberrations changing across the field on phone hardware at any decent speed is pretty challenging...

If you're interested in the equivalent of "backprop through zemax" there are a few projects going on to jointly optimize optical designs with the image processing, e.g. check out: https://vccimaging.org/Publications/Wang2022DiffOptics/

I've been working on something similar, although I'm more interested in replicating the effects of existing lenses than designing new ones: https://x.com/dearlensform/status/1858229457430962318

PBRT 3rd edition actually has a great section on the topic but it's one of the parts that wasn't implemented for the GPU (by the authors, anyway): https://pbr-book.org/3ed-2018/Camera_Models/Realistic_Camera...

Colour in the Photoshop/gamedev world is often handled pretty casually, but if you're interested the moving picture world gets a lot more rigorous about it and there's tons of documentation around the ACES system in particular: https://github.com/colour-science/colour-science-precis https://acescentral.com/knowledge-base-2/

As you suggest storage in linear 16-bit float is standard, the procedure for calibrating cameras to produce the SMPTE-specified colourspace is standard, the output transforms for various display types are standards, files have metadata to avoid double-transforming etc etc. It is complex but gives you a lot more confidence than idly wondering how the RGB triplets in a given JPG relate to the light that actually entered the camera in the first place...

I've worked on a few bloody moments in film VFX and we certainly looked at some horrible reference of wounds, including headshots from pathology reports. I don't think I suffered much beyond the odd sigh but it was only for a couple weeks at a time. Someone went and talked to a trauma nurse about how much blood would be expected for various rounds and whatnot, and I did imagine them not being terribly impressed at why we wanted to know. Even though it was for a rather non-violence-glorifying film in that case.

At some point someone did tell me something that helped though: think of and describe what you're looking at using food terms instead of anatomy ones - talk about a blood fluid sim as being "too much like tomato juice not enough like gravy", textures in terms of mince/steak etc rather than human tissue types. I also labelled render passes like that, made conversations sound a lot less gross and more removed from reality, particularly to passers-by working on other shots...

A lot of that group's publications are listed here, many involving optical flow: http://web.archive.org/web/20161014025823/http://gpu4vision....

Maybe this one from 2007: https://www-pequan.lip6.fr/~bereziat/cours/master/vision/pap...

I'm actually not sure much ML was involved here - depends where you draw the line I guess, but denoising and interpolation for restoration typically use more traditional wavelet and optical flow algorithms. The work for this was done by Park Road Post and StereoD, which are established post-production facilities using fairly off-the-shelf image processing software. The colorisation likely leant heavily on manual rotoscoping, in the same way that post-conversion to stereo 3D does.

I'd love to hear otherwise but I'm not aware of any commercial "machine learning" for post-production aside from the Nvidia Optix denoiser and one early beta of an image segmentation plugin.

Perceptual uniformity is in some ways opposite to the linearization suggested above - the L* component of CIELAB is much more like the gamma-encoded values of sRGB than a linear light measure.

It seems tough to come up with hard and fast rules for whether to mimic the linear physical processes, or work in a perceptual space more like the human visual system. I'd love to hear about more rigorous work in this area - most things I read have boiled down to "this way works better on these images".

It's interesting for example that using Sinc-type filters to resize truly linear data, like that from HDR cameras, usually gives rise to horrible dark haloing artifacts around small specular highlights, despite that being the most "physically correct" way to do it. Doing the same operation in a more perceptual space immediately sorts out the problem.

Digital cinema uses wavelet compression - intra-frame only JPEG2000 at hundreds of Mbps. It seems that at high resolution and bitrate it actually performs similarly to or better than h264, e.g. this paper and its references: http://alumni.media.mit.edu/~shiboxin/files/Shi_ICME08.pdf

That's exactly how it's normally done - the offline editor works with low bitrate HD files, then gives the edit list to the online team who re-conform from the full resolution stuff.

There's always an impetus to get higher resolution and quality at the offline stage though, as people get used to improving technology. It wouldn't be outrageous to have a 4k offline now, because people with 4k TVs at home will say "Why can't we? I want to see everything!", even if it doesn't particularly enhance the editor's work.

It would still be done with lower bitrate files, because original 4k material from digital cinema cameras is beyond what most machines are happy with.

Shamefully I can't remember which wahwah I used - I was treating the audio with both Audacity and Pro Tools as well as sox though...

Similar results - converting to a square image at 8bit then lowering JPEG quality just gets noisy in an uninteresting way:

    sox mo.wav -e unsigned -b 8 -c 1 -r 48k mo.raw
    bytes=`stat -f %z mo.raw`
    width=`echo sqrt\($bytes\) | bc`
    square_bytes=`echo $width \* $width | bc`
    dd if=mo.raw of=mo_square.raw bs=$square_bytes count=1
    gm convert -depth 8 -size ${width}x${width} gray:mo_square.raw -quality 50 mo_square.jpg
    gm convert mo_square.jpg gray:mo_square_jpg.raw
    sox -e unsigned -b 8 -c 1 -r 48k -t raw mo_square_jpg.raw mo_jpg.wav

Nice script :) I did a similar thing by hand with graphicsmagick and sox a couple years ago - the most fun result was from wahwah, pretty much a wide bandpass filter with centre frequency varying cyclically over time: http://lewisinthelandofmachines.tumblr.com/post/59405537096/...

Strikingly similar to the low-discrepancy sequences used as sampling patterns in modern ray tracers - and quite a similar role, really:

https://en.wikipedia.org/wiki/Halton_sequence

https://books.google.co.uk/books?id=DirOQ_PELlgC&lpg=PA999&o...

The IE11 result makes me weep here, given the popularity of it - thanks for doing these tests. The amount of work we put into maintaining correct colour through the production chain seems increasingly wasted...

FYI I tested Safari 9.1.1 (11601.6.17) and it failed on a whole bunch of the tests, including, frighteningly, the untagged HD 709 H264 :(

Correct! But for wider gamuts (and higher-contrast displays), the benefit becomes more obvious in the lack of banding.

This is kinda happening - the "UHD Alliance Premium Certified" spec for TVs mandates 10-bit from the input to the panel. It's a shame the UHD Blu-ray standard doesn't mandate 10-bit, thought hopefully most will use it :) Dolby Vision mandates 12-bit mastering and delivery, though it sounds like 10-bit connections to the panel can be considered acceptable...

The antialiasing linear vs. gamma debate is an interesting one - check out this conversation, wherein nobody could figure out a reasoned method other than "sometimes AA in sRGB looks good"... https://twitter.com/rygorous/status/512371399542202368

10-bit with a log or gamma encoding is widespread in film and video work, and I've never known a banding problem, even with purely generated gradients. The Rec. 2020 UHD standard does recommend 12-bit gamma-encoded to deal with the ludicrously wide gamut.

For HDR, a PQ (perceptual quantisation) encoding curve is already standardised by SMPTE - page 8 in these slides goes through the process of how it was worked out, right from human visual system basics: https://www.smpte.org/sites/default/files/2014-05-06-EOTF-Mi...

Given the abundance of log or gamma encodings for display imagery you might wonder why true linear 16-bit float is so common in CG production - why not log encode those 16 bits and get loads smoother gradients? Maybe the answer is that during production those linear files often also encode non-image data like vertex positions and normals, and perceptually "good" quantisation of those could lead to unexpected precision problems...

Shake is more complete than Natron still - and some parts of Shake are still world class, the spline-based warping for example is still on par with Nuke and Flame, and the included versions of Keylight and Primatte haven't been hugely improved on since. And Smoothcam is super handy.

I have had to take increasingly drastic measures to keep it running on modern Linuxes though, and on OS X it sadly now seems impossible :(

I was always curious how much use that Spirit support saw - it must have been a ton of work, but it seemed like it was added just at the precise moment when everyone switched from running the film in real time during a session to scanning to DPXs beforehand!

I occasionally see Spirit 2k's destined for the dump and wonder what would be entailed in running one. Since they were a little notorious for poor frame-to-frame registration I wonder if that will get worse as the mechanics age.

It's a strange time for telecines, with so many companies having disappeared but newer, much more compact designs appearing, for archival scanning presumably. If you haven't discovered it already the archives of the TiG mailing list are a great resource for TK stuff - e.g. this thread about the new Blackmagic/Digital Vision/MWA/Kinetta scanners... http://tig.colorist.org/pipermail/tig/2016-February/thread.h...

I'm interested to know how they're interfacing with the Adobe apps. I hooked up trackballs to Levels in Photoshop using the official "Connection SDK" a couple of years ago and the latency felt terrible, as far as I could tell due to Photoshop's response time. Hopefully something has improved there in the interim... http://lewisinthelandofmachines.tumblr.com/post/60137039416/...

Seems a shame there isn't a standard for mapping hardware to generic UI controls, the way MIDI is ubiquitous in audio production. All video apps support control surfaces of some kind but Avid/Tangent/Filmlight/Davinci/Lightworks panels all use their own closed protocols and I'm not aware of any of them having been reverse-engineered :(

With modern film stocks and scanners Super8 can look remarkably good. It'll be interesting to see how Kodak's scanning service compares to scans like these, both of which were shot with another new Super 8 camera, the Logmar Digicanical:

https://vimeo.com/129700087

https://vimeo.com/groups/super8/videos/87243287

I can't imagine btdigg can scrape the whole DHT, but I think the idea is that the traffic they see by running many "fake" nodes is proportional to the whole, because the DHT spreads all traffic around fairly equally? I'm presuming they're using a method similar to the one presented here: https://www.usenix.org/legacy/event/woot10/tech/full_papers/...

Aha, the long .torrent file it drops matches the magnet link too. The MD5 of AF.dat from that page doesn't match what I've got but maybe it gets modified...

Interesting, whilst downloading the bittorrent client ID strings were all "LT/1.0.3.0", maybe a connection to the Lineage malware from that page?

For the curious, the magnet link is: magnet:?xt=urn:btih:a4a75d2e4095d457467777673e96cd331575b511&dn=AF

file(1) has nothing to say about it but at a glance it doesn't look like a uniform encrypted blob...

Nuke works in scanlines like this, and can process a whole tree of operations only loading the input lines necessary for the current output row. The SDK docs explain the architecture somewhat: https://www.thefoundry.co.uk/products/nuke/developers/90/ndk...

It's not great but it is slowly getting better as the big studios open-source their inhouse formats - Disney, Sony Imageworks and ILM are all doing so and Alembic, OpenVDB, OpenSubdiv, OpenEXR, and Ptex are all widely supported now. The process of opening up things people are already using seems to be working out much better than previous committee- or vendor- driven efforts like FBX, X3D or AAF...

That said, I spent yesterday trying to figure out how to get Leica laser scan data into Houdini, and the only way is to convert to ASCII CSV, so we've got a way to go. Text files are great for debugging, but when you've got 300 million points you pine for the I/O speed of a binary format :/