1,685 karma · joined September 17, 2015
The blockchain adds nothing here, and creates a layer of complexity that makes it more brittle.
In general though, I believe solving real-world problems with whatever tech is available creates the most value. (And I mean real value, not some numbers on a ledger that don't mean anything at all.) That doesn't mean tech for tech's sake isn't interesting.
If I heard a single use case where blockchain technology actually created real value in the world, and was better than other alternatives, I'd listen. But it has to create real value, based on the real world and not some fantastical notion of what money is or governments are for.
I think we skeptics want a grounded narrative, and we're stuck hearing a thesis statement and nothing to back it up.
Blockchain? The money flowing in is substantially from retail investors. And that's the problem. This is why we have regulations.
It's a Ponzi scheme. There's no inherent underlying value, its "market cap" is strictly a function of more dumb money flowing into the system. Occam's razor, this is the most likely explanation for everything in the crypto space; the model makes sense and explains a lot more than "a new financial asset" hokum.
Anecdotally, I have heard people applying different timescales for the hype cycle for different technologies. Makes the pattern that much easier to match.
Except for bitcoin's current "monetary" uses, like money laundering and tax evasion.
This is once again better served by a database. Diplomas are inherently centralized and issued by a trusted authority.
Guess we gotta keep brainstorming for use cases -- that's how technology development works, right?
The quality of global shutter cameras is simply nowhere near the quality of rolling shutter cameras, though there are more acceptable options showing up these days, they still typically have lower dynamic range and generally worse specs.
The bigger rental houses make all their money on other things -- the ROI on sandbags, sound blankets, even consumables is ridiculous compared to cameras. (Granted I'm blending grip and camera, but some houses do both.)
You're also competing with owner/operators, and even in-house studio operations. It doesn't scale -- relationships matter and it's highly regional. Compared to other ways to put your capital to work, it's not a great investment.
Don't remember the sources for the anecdote, sorry. But the wikipedia page corresponds:
As for quality, the highest of high end lenses are much sharper for a given f-stop, and sometimes have higher t-stop ratings, than cheaper lenses you can get. They will look consistent through all focus settings, and won't "breathe" (zoom while focusing). The lenses also come as a matched set, so you can switch between focal lengths and the images look the same -- no color shift or aesthetic change. This puts a huge number of constraints on the lens designer, requiring different materials and manufacturing processes. Cinema lenses aren't irrationally priced, even if the demand is driven by aesthetics.
And yes, modern lens designers could probably recreate classic lenses. No, it's not economically viable, even at outrageous prices. (Because you can digitally fake most of the obvious artifacts, and the few people who really care about the in-camera look aren't enough to justify the engineering cost.)
There may also be a minor materials issue; some of the glass used in very old lenses was pretty exotic, some were even mildly radioactive. It's possible those glasses are not available since there are better replacement options in glass catalogs now. But this is just speculative, I haven't tried to source thorium glass. (Yet?)
When they don't, like the Canon 5D mark ii when used in video mode (which just threw out most of the sensor's pixels to get to 1080p), you get aliasing producing moire patterns like this:
Governments that borrow in their own currency reduce debt best via growth, since debt is stated in terms of GDP.
On the other hand, people who are short dollars — debtors with fixed debt — are helped by inflation. Have a mortgage, student loan payments, even car payments that aren’t linked to inflation? Then inflation does reduce your debt burden.
Inflation hurts those groups that are long monetary assets — those who own fixed debt, or cash. Its effects on other asset prices, like stocks, are less direct. For example, stock prices reflect changes in growth in the economy that may be hindered by high inflation (or high interest rates).
Simple models are great except when they’re wrong.
But yes, it's far more realistic than your simple mesh-and-texture; the video shows it handles reflections and transparency. When NeRF came out (~2020?) it was a major step forward in light field quality and this technique appears to maintain that quality.
The first is metamerism, or more generally the fact that we don't observe wavelength directly. Many of the blue colors we can perceive in the real world have a lot of "green" and "red" wavelengths in them too; the cones that detect green and red will still fire for those collections of wavelengths, but the blue-detecting cones will fire faster. We perceive "blue" through higher level processing of those signals; furthermore, it's not a 1:1 mapping.
More importantly: most light we see isn't spectrally pure, so chromatic aberration is not as significant as those charts seem to indicate; rather than multiple distinct planes of good focus, you really have a general region that has a minimum spot size. For most human eyes that spot size is pretty small; errors in the human eye are going to be a larger factor than chromatic aberration for most people.
Also significant: most chromatic aberration people talk about in lenses is lateral chromatic aberration, which can be significant in biological eyes — it's supposedly why herbivores with wide fields of view have horizontal irises! — but for human color vision, most of our acuity is in the foveal region, for which LCA is vanishingly small. The magnitude of LCA is a function of angular distance from the chief ray (~foveal). Axial chromatic aberration has a much smaller effect and generally only affects your minimum spot size, but as mentioned before biological eyes are not accurate enough for this effect to dominate.
Now, what the article nearly gets right: there are examples of light you can't focus on: short wavelength light that is spectrally pure (and sufficiently distant) could be 'unfocusable' for certain eyes. I can report with high confidence this is true for my eyes, and likely for a lot of other people. You can see this with blue laser light, some violet-hued LEDs (not red + blue mixed), or even the visible spectrum from mostly-UV sources. I find that the light itself will appear to have a halo and not be particularly sharp; for me, the perceptual effect is not quite like it being blurry, because the shape of the 'blur' is affected by my eye's aberrations, so it tends to have a 'spiky' bokeh. YMMV.
Finally, there is a truth about human color vision that the tests here seem to get close to: we mostly perceive sharpness or acuity in images as a function of luminance, not chrominance. (Modern image and video formats exploit this extensively.) In fact, the equation to derive luminance from chrominance values models this explicitly; it varies based on color space, but here's the equation for sRGB:
Y = 0.2126R + 0.71522G + 0.0722B
Where Y is luminance, and R, G, and B are the individual color channel values (source: https://en.wikipedia.org/wiki/Relative_luminance). Note the coefficient for blue suggests that only ~7% of our brightness perception derives from the blue channel. This is the source of chroma-based data compression, where we throw away data in the chrominance channels but maintain it in luminance (see YCrCb coding), which is also the reason why blue seems to matter less when doing per-channel blurring: it simply contributes less to the perceived luminance value due to the mechanics of human color perception.
That said, yes, it's odd they're posing this as a way to extend game engines, when this entire line of research is interesting (to me) for its potential to be a generalized holographic codec. I'm not qualified to evaluate if this is generalizable, but I would hope these results also indicate a possible way forward for camera-derived footage, where there are not acceptable approaches. But to your point:
>As for the scenes in this demo, I'm pretty sure that structure from motion + mesh reduction + normal mapping can reconstruct the original images almost pixel-perfect, and at 200+ fps on modern GPUs.
I don't agree, we've seen attempts to do generalized photogrammetry as a holographic codec, and the results are poor. Microsoft's HoloCapture is probably the best commercial example, and, well... it ain't pretty. (https://holocap.com/) I think the target should be visual fidelity at parity with good 2D codecs. So there's a long ways to go.
The paucity of data sets may be underselling how hard this is and how advanced NeRF (and other approaches like MPI) are relative to advanced photogrammetry. In particular, approaches that extend to video and work well with the human face are very desirable, but there aren't a lot of datasets to test against. And yet these approaches _are_ significant improvements over photogrammetry; I'm not at liberty to share, but I've seen a demo comparing pure photogrammetry to a neural renderer, and the difference is night and day.
But let me agree with you: yes, we should throw these new representations at difficult subject matter, including reflections, translucency, refraction, subsurface scattering, etc, and see how they do. The original NeRF paper had some wild results for refraction, for example, which hadn't been demonstrated before. And when possible, I'd love to see them with video images of people.
I think we're only a few years away from a functional holographic codec, which is pretty exciting.
Sadly, you need to have significantly higher light field resolution to do an 'acceptable' job of approximating lens aberrations, and in the end it probably wouldn't look better than a 2D filter approximating it. Nonetheless I like the idea.
https://expresso.com/ https://www.virzoom.com/
Expresso is more commercially-oriented, but iirc they lost high eight figures starting back in the 2000s. Virzoom only burned something like seven figures so far, working against the added drag of VR adoption.
And then there's Zwift (https://www.zwift.com/), which doesn't carry the hardware costs and was recently valued at ~$1 bn+.
If I'm playing armchair CEO, it seems like you do better off by leveraging the existing hardware ecosystem and just building a content / network play. Better to gift Apple / Google 30% than try to build, launch, and support hardware.
Also, it's really hard to get enough users to support a fun enough game library; the overhead of custom content development is way, way too high unless you've got, I dunno, tens of millions of users? Everyone's thinking of Peloton, but they adopted the highly successful spin class model to the home. The change in behavior was staying home rather than going to the gym, not starting exercise instead of playing video games. And full disclosure, yes: I was skeptical of Peloton before they launched, too.
https://www.wunderparc.com/ https://www.prometheanai.com/
I haven't tried either one out; someone in my industry pointed them out to me recently.
But be careful; I've noticed that if you use too much baking soda, you can get odd flavors — like squash tasting like pretzels.
Phase detection is not the same thing at plenoptic; the article implies that phase detection operates in the same way as a microlens array. It doesn't; microlens arrays for plenoptic imaging have many pixels underneath, and perform different operations on them. This line:
"With only two pixels in one, there's still enough to calculate a fair optical depth of field map without having a second camera like everyone else."
This is categorically false. You need dozens of pixels underneath a microlens array to do digital refocusing; phase detection is not remotely the same thing.
Later, the author conflates microlens-based refocusing with camera-array based plenoptic imaging. These are wildly different disciplines; the work done in VR does not include microlens arrays anywhere. Not any. But they do include large arrays of cameras (see volumetric work from Microsoft HoloCapture, Intel's capture studio in Manhattan Beach, and my own company Visby's work — cameras at Radiant Images in LA... all arrays of cameras, no microlenses anywhere).
"Apparently, if you take only one central pixel from each cluster and build the image only from them, it won't be any different from one taken with a standard camera." This is false; taking the central pixel will give you an aliased image (or an image as though taken with a very, very high f-number lens, if the pixels correspond to the same zone of the aperture). To make an image as though you didn't use a microlens array, you sum all the pixels under the microlens to produce a macropixel. This is why light field cameras have lower effective resolution than traditional cameras. The remainder of the speculation about 'sneaky plenoptic JPEGs' has no basis in light field imaging; refocusing is not achieved by throwing away (or binning) pixels, it's done by summing different sets of pixels. Lytro founder Ren Ng's PhD thesis was about a way to do it faster by using the Fourier Slice Theorem. (Beautiful paper, if you have time to read it.)
Google did not "buy and kill Lytro." Lytro's assets were sold and Google bought some of them, but none of that tech made it into the Pixel line of cameras, period. The Google portrait mode is done using phase detect pixels as a cue for segmentation, as discussed in the article this blog links to. Nothing plenoptic required, and certainly no Lytro tech there. The blog post implies that the Pixel is using Lytro-like techniques rather than phase detect pixels to hint at segmentation for synthetic blur.
I have never heard of plenoptic cameras being used for image stabilization. In my personal experience (5 years of stabilizing cinema cameras for two different companies), virtually all stabilization issues derive from rotational blur, not translational movement. Your ability to stabilize translational movement would be limited to the aperture size, and translations of that magnitude have vanishingly small effects on the image unless the objects are very, very close to the lens (like macro). I have no idea where the blogger came up with this idea but there is no reference provided and I have never heard of it (nor would it work, for the above reasons).
The section "Fighting the Bayer Filter" miscasts the challenges of a Bayer pattern under a microlens; naively summing adjacent pixels under other microlenses would blur an image. You do get a boost to dynamic range from the synthesized images — you're binning a bunch of pixels to create your image, so you reduce noise as 1/root(n) in the number of samples. That could lead to better color in dark, noisy areas, but it will not increase the gamut of your sensor.
It goes on... but since this article was already off the front page by the time I posed my comments, this is probably enough.
(As an aside, Jacques — I really enjoy your writing!)