Physically based rendering from first principles
imadr.me
imadr.me
Regarding the 'first-principles' discussion, it is a relative term in usage despite the name. Explaining from a layer of abstraction below normal explanations.
Someone explaining how a computer works starting with machine instructions is doing first-priniciples, even if they wont explain details of atoms. The literal meaning would be almost impossible to implement, for instance none of Newton's work would be first principles given today's knowledge. Similarly, current physics can be subsumed in future principles.
In fact, I'd like to argue that you shouldn't learn things from first principles, at least in the beginning. Despite the article not being from first principles, it does illustrate some of the problems of first principles: they are pedantic. Everything stems from first principles so they have to be overly pedantic and precise. Errors compound so a small error in one's first principles becomes enormous by the time you look at what you're actually interested in. Worst of all, it is usually subtle, making it difficult to find and catch. This makes them a terrible place to begin, even when one already has expertise and is discussing with another expert. But it definitely should not be the starting place for an expert to teach a non-expert.
What makes it clear that the author isn't a physicist is that they don't appear to understand the underlying emergent phenomena[0]. It's probably a big part of why this post feels so disordered. All the phenomena they discussed are the same, but you need to keep digging deeper to find that (there's points where even physicists know they are the same but not how or why). It just feels like they are showing off their physics knowledge, but it is well below that which is found in an undergraduate physics degree[1]. This is why you shouldn't start at first principles, its simplicity is too complex. You'd need to start with subjects more complicated than QED. The rest derive out of whatever a grand unified theory is.
But as someone who's done a fair amount of physical based rendering, I'm just uncertain what this post has to do with it. I would highly recommend the book "Physically Based Rendering: From Theory To Implementation" by Pharr, Jakob, and Humphreys that the author says the post is based on. It does a much better job at introducing the goals and focusing on getting the reader up to speed. In particular, they define how the goal of PBR is to make things indistinguishable from a real photograph, which is a subtle but important distinction from generating a real photograph.
That said, I still think there's nice things about this post and the author shouldn't feel ashamed. It looks like they put a lot of hard work in and there are some really nice animations. It's clear they learned a lot and many of the animations there are not as easy as they might appear. I'm being critical but I want them to know to keep it up, but that I think it needs refinement. Finding the voice of a series of posts can be quite hard and don't let stumbles in the beginning prevent you from continuing.
[0] Well that and a lack of discussion of higher order interference patterns because physicists love to show off {Hermite,Laguerre}-Gaussian mode simulations https://en.wikipedia.org/wiki/Gaussian_beam#Higher-order_mod...
[1] In a degree you end up "learning physics" multiple times. Each time a bit deeper. By the end of an undergraduate degree every physicist should end up feeling like they know nothing about physics.
Let's suppose the aim of the article was indeed to learn PBR from first principles, what would it look like? Quantum electrodynamics?
I think there is merit in exploring different physical models for fun and scientific curiosity (like I mentioned in the first chapter). I (personally) feel that it's boring to just dump equations like Snell's law without exploring the deeper meaning behind it. I also feel that it's easier to grasp if you have some surface knowledge about more complex physical models.
I agree however that I probably made many mistakes since I didn't study physics, I'd appreciate any feedback to improve that.
I dislike "Physically Based Rendering: From Theory To Implementation", I personally think that the literate programming approach of the book is way too confusing and disorganized. I prefer the SIGGRAPH talk by Naty Hoffman[0]
Well given the title I at least expected the article to explain or derive things like why and how metals and their alloys have the color (wavelenght-dependent complex index of refraction) that they do, why and how say quartz crystals have different colors, birefringence, fluorescence (makes T-shirts appear extra bright) etc.
And there is no mention of the recording process. Are we simulating good old film with its silver crystals of various sizes? Different film stock is known to have very different looks due to their chemistry.
Or a digital camera sensor with its quantum and thermal noise, bayer filter and a rolling shutter causing those funny-looking propeller pictures?
Not knocking the article, but given the title it fell well short of my expectations going in. That is, I was wondering how on earth anyone had managed to do all that.
Interesting. Personally it's by far the best programming related book I've read. I didn't mind the literal programming, and I loved how it dove fairly deep into the math and physics but also into the details of implementing the math.
The latter being important as there are can be so many gotchas when implementing math.
Something like that, yes. A truly from-first-principles treatment of photon-surface interactions would involve an extremely deep dive into quantum numbers, molecular orbitals, solid state physics and crystal lattices (which are metals), including a discussion about how electron waves superpose to produce various conduction/valence bands with various band gaps, and then discuss how photons interact with these bands.
If you had to teach an alien from another universe physically based rendering:
- In an exhaustive manner and,
- You're only allowed to explain something if it derives from something more "fundamental" until we reach the most comprehensive physical models we have
How hard would be the math behind it for example? Because realistically in my article the hardest piece of math is a very basic integral
Could I for example start reading these Feynman lectures[0] and get up to speed on everything there is to know about photon-surface interaction?
The issue is the sheer complexity of micro systems, and the unintuitive nature of going deeper than 'EM wave reflects off electrons'.
Consider metal-light interaction. Exactly how does a visible-light EM wave interact with a conduction band of superposed free valence electrons? How does the massive superposition elevate each valence electron up energy levels? Why do only metallic and semi-metallic crystals have no band gap? Why are electrons filled in the order of s, p, d, f, g, h orbitals? Why do these orbitals have these shapes? Why are electrons so much less massive than protons and neutrons? Why does the nucleus not tear itself apart since it only contains positive and neutral particles? Why are protons and neutrons made of three quarks each, and how does the strong interaction appear? Why are the three quarks' mass defect so much more than the individual masses of each quark? How does the mass-energy equivalence appear? Why does an accelerating electric charge produce and interact with a magnetic field, and thus emit EM radiation? What is mass, charge, and magnetism in the first place?
Each question is more 'first principles' than the last, and the answers get more complex. In these questions we have explored everything from classical EM, to solid state physics, to quantum electro- and chromodynamics, to particle physics and the Standard Model, and are now verging on special and general relativity.
It sounds like the OP is saying you could tackle the article if you just know a little high school calculus and trig, and you're here saying that you need years of post-secondary mathematical education. I think you're making his case that he's dramatically simplified understanding how to compute these things for someone who doesn't have a post-secondary education.
you don't need to go to quantum physics or general relativity for everything, that would mean that a description from first principles is impossible for any field that doesn't (yet) directly connect there like biology, psychology, you name it. which may be technically correct but clearly not in the spirit of the idea.
in general this article has a quite surface level focus and narrow scope. there isn't even a description how the most common physical rendering primitive ever, a ray of light, is related to electromagnetic waves in mathematical terms. the article is not very mathematical in general which is disappointing.
> Let's suppose the aim of the article was indeed to learn PBR from first principles, what would it look like?
All this is well beyond my expertise btw, despite having worked in optics. If you want to see some of this complexity, but at a higher level, I'd highly recommend picking up Jackson's Elecrtodynamics book. That's that canonical E&M book for graduate level physics, Griffith's is the canonical version for undergraduate (Junior/Senior year). Both are very well written. I also really like Fowles's "Introduction to Modern Optics", and it is probably somewhere in between (I read it after Griffiths).
I am in full agreement with you that having deep knowledge makes a lot of more shallow topics (and even many other deep topics) far easier to grasp. But depth takes time and it is tricky to get people to follow deep dives. I'm not trying to discourage you here, I actually do encourage going deep, but just noting how this is a tricky line and that's why it is often avoided. Don't just jump into the deepend. Either wade people in or the best option is to lead them in so they don't even recognize they're going deep until they're already there.
> I dislike <PBR Book>, I personally think that the literate programming approach of the book is way too confusing and disorganized
So, who is your audience?
This is just my opinion, so you have to be the real judge; but I think you should leverage your non-expertise. One of the hard things when teaching is that once you understand something you quickly forget how difficult it was to learn those things. We quickly go from "what the fuck does any of this mean" to "well that's trivial" lol. You referenced Feynman in your blog post and most important thing I learned from him is one of the best tools for learning is teaching (I've given way too many lectures to my poor cat lol). It forces you to answer a lot more questions, ones you normally would table and eventually forget about. But at your stage it means you have an advantage, that the topics you are struggling with and have overcome are much more fresh. When learning things we often learn from multiple sources (you yourself shared overlapping material), and that's because multiple perspectives give us lots of benefits. But at this point, don't try to be a physicist. If you want to work towards that direction, great! If you don't, that's okay too. But let your voice speak from where you are now.
Reading your blog post and poking through others, there's a "you" that's clear in there. Lean into it, because it is good. I like your attention to detail. Like in your Ray Marching post how you just color code everything. Not everyone is going to like that, but I appreciate it and find it very intuitive. I'm a huge fan of color coding equations myself and make heavy use of LaTeX's annotate-equations package when I make slides.
But I think looking at this post in isolation the biggest part to me is that it is unclear where you're going. This is a problem I suffer from a lot in early drafts. An advisor once gave me some great advice that works really well for any formal communication. First, tell "them" what you're going to tell them, then tell them, then tell them what you told them. It's dumb, but it helps. This is your intro, it is your hook. I think there's places for these ideas but early on they almost feel disconnected. This is really hard to get right and way too easy to overthink. I think I can help with a question: "What is your thesis?"/"What is your main goal?" Is it "learn how our human eyes capture light and how our brains interpret it as visual information"? Or is it "Physically based rendering from first principles". Or even "learn how to create physically realistic renderings of various materials." These are not exactly the same thing. When I'm struggling with this problem it is because I have too much to say. So my process is to create a "vomit draft" where I just get all the things out and it's going to be a mess and not in the right order. But once out of my head they are easier to put together and in the right order. After your vomit draft check your alignment. What is most important and what can be saved? What's the most bare bones version of what you need to communicate? Build out of that.
I do really think there's a good blog post in here and I can see a lot of elements that suggest a good series may come. So I do really encourage you to keep going. Look at what people are saying they like and what they dislike. But also make sure to not take them too literally. Sometimes when we complain about one thing we don't know our issue is something else. What I'm saying is don't write someone else's perfect post, write your post, but find best how to communicate what you want. I know I've said a lot, and I haven't exactly answered all your questions, but I hope this helps.
[0] There's a side note here that I think is actually more important than it appears. But the thing is that there's a weird relationship between computation and accuracy. I like to explain this looking at a Taylor Series as an example. Our first order approximation is usually easy to calculate and can usually get us a pretty good approximation (not always true btw). Usually much more than 50% accurate. Second order is much more computationally intensive and it'll significantly increase your accuracy but not as much as before. The thing is accuracy converges much like a log-like curve (or S-curve) while computation increases exponentially. So you need to make these trade-offs between computational feasibility and accuracy. The most important part is keeping track of your error. Now, the universe itself is simple and the computational requirements for it are lower than it takes us to simulate but there's a much deeper conversation about this that revolves around emergence. (The "way too short" version is there's islands of computational reducibility) But the main point here is this is why you should typically avoid going too low quickly, because you end up introducing too much complexity all at once and the simplicity of it all is masked by this complexity.
I strongly agree that teaching is absolutely the best tool for learning. I wrote this article in part because I got inspired by the "What I cannot create, I do not understand" quote by Feynman.
I agree that the article is disorganized, and it's not only a feeling: it literally is! I had to shuffle around parts of the chapter about radiometry because I couldn't find the right place for it. I was kind of in a rush because I submitted this article as part of 3blue1brown's summer of math exposition.
I find it interesting that between the 3rd and 4th edition of pbr book, chapters have been reorganized too. Monte Carlo Integration has been moved to an earlier chapter before radiometry, reflection models..etc which I found confusing, but I suppose the authors had a good reason to do that. So I have a lot to learn on how to improve my writing and how to organize ideas into something coherent.
> But the thing is that there's a weird relationship between computation and accuracy. I like to explain this looking at a Taylor Series as an example. Our first order approximation is usually easy to calculate and can usually get us a pretty good approximation (not always true btw). Usually much more than 50% accurate. Second order is much more computationally intensive and it'll significantly increase your accuracy but not as much as before. The thing is accuracy converges much like a log-like curve (or S-curve) while computation increases exponentially.
But I think first I'd just do some Taylor or Fourier expansions of some basic functions. This can help you get a feel of what's going on and why this relationship holds. The Taylor expansion one should be really easy. Clearly the second derivative is more computationally intensive than the first, because in order to calculate the second derivative you have to also calculate the first, right?
Mind you there are functions where higher order derivatives are easier to calculate. For example, the 100th derivative of x is just as easy to calculate as the second. But these are not the classes of functions we're usually trying to approximate...
So maybe a classic example of this is the infamous 80/20 rule. You can read about the Pareto Principle[1] which really stems from the Pareto Distribution, which is a form of a Power Distribution. If you're looking at the wiki page for the Pareto Distribution (or Power Law), you'll see the shapes I'm talking about.
A real life example of this is when you train a machine learning model. Let's take accuracy for just simplicity. Let's look at PyTorch's example on using Tensorboard since that includes a plot at the very end[2]. Their metric is loss, which in this case is the inverse accuracy. So Accuracy is 0-100 (0 to 1) where higher is better, loss is just 1-accuracy, so 0 means perfectly accurate. From 0-2k iterations, they went from 1 to 0.6 (a 0.4 gain). Then at 4k iterations they are at a 0.4 loss (a 0.2 gain over 2k iterations). You see how this continues? It is converging towards a loss of 0.2 (accuracy = 80%). This is exactly what I'm talking about. Look at your improvements over some delta (in our case loss/(2k iterations)). It's a second order effect here, meaning it's non-linear.
This nonlinearity shows up everywhere. Going back to the 80/20 rule, it is often applied to coding. 80% of code is written using 20% of the time, but 20% of the code is written with 80% of the time. This should make sense as there are different bottlenecks. We'd be naive to just measure by lines of code (see [0]). A lot of time is spent on debugging, right? And mostly debugging just a few key areas. The reason this is true can derive from a simple fact: not all lines of code are equally as important.
So the other example I mentioned in the previous comment is Fourier Series[3]. That wiki has some nice visualizations and you'll be able to grasp what I'm talking about from them. Pay close attention to that first figure, the middle plot (image 2/17). These are different order approximations to a square wave. Might be hard to see, but as the more complex the wave (higher order) the better approximation you get to that square wave. Pay close attention to the calculations. Do a few yourself! How much work goes into calculating each term? Or rather, each order of approximation. I think you'll get the sense pretty quickly here that every higher order calculation requires you to also do the lower order ones.
As a more realistic example I am the creator of a state of the art image generator (I won't say which one to maintain some anonymity). When training my model the score quickly improves and really only a small amount of time. This training run took approximately 2 weeks wall time (what the clock says, not the GPU). Most of the improvement (via metric) took place in the first 6hrs. I was >90% of the way to my final score within the first day. If you look at the loss function in full, almost everything looks flat. But if you window it to exclude the first 24hrs, the shape reappears! There's a fractal nature to this (Power Distribution!). To put numbers to this, my whole run took 1M iterations and my final score was ~4.0. My first measurement was at 5k and was 180. My next measurement was at 25k and at 26. 15@50k, 9@100k, 6.8@200k, 5@500k, and so on. This is very normal and expected. (Then there's the complexity of [0]. Visually the images improved too. At 5k they were meaningless blobs. By 100k they had the general desired shape and even some detail appeared. By 500k most images resembled my target. At 800k I had SOTA but had could tell things were off. By 1M I thought there was a huge visual improvement from 800k but this is all down to subtle details and there are no measurements that can accurately reflect this)
I am happy to answer more but you're also asking about a complex topic with a lot of depth. One I absolutely love, but just giving you a warning :)
[0] The super short version is no matter what measurement you take you are using a proxy. Even a ruler is a proxy for a meter. It isn't exact. When measuring you approximate the measurement of the ruler which is an approximation of the measurement of a meter. This case is typically very well aligned so the fact that it is a proxy doesn't matter much (if you include your uncertainties). This isn't so simple when you move to more complex metrics like every single one you see in ML. Even something like "accuracy" is not super well defined. Go through a simple dataset like CIFAR-10 and you'll find some errors in labels. You'll also find some more things to think about ;) Table this for now but keep it in the back of your head and let it mature.
[1] https://en.wikipedia.org/wiki/Pareto_principle
[2] https://docs.pytorch.org/tutorials/intermediate/tensorboard_...
If you don't mind, could I talk about it with you more over email? My email address is listed in my profile.
You couldn't have been any closer. I emerged feeling extremely stupid after doing physics at uni. My rather mediocre marks didn't help...
I don't fear the man who knows more than me, I admire them and only have something to gain by admitting my ignorance to them. But I do fear the man who believes he knows more while in reality he doesn't. Nothing productive comes from them as they don't even recognize the journey we are both on.
> How much more “first principles” could this article be!?
As to your actual question(?) of "how more fundamental can it be"? Well, the answer is a lot. They barely scratched E&M theory. I specifically mentioned that even a undergraduate in physics would be exposed to much more fundamental aspects. Likely even before their junior year.
But if you're asking "how much more fundamental should* it be" well most of my comment is arguing that it should not be. I argued that it generally isn't a good idea to start from first principles, and I'll even argue that it probably isn't a good idea to start there even if they are in quotes.
Do you really want to start learning physics from String Theory? You could, but it isn't a great idea. Even if you replace ST with an alternative proposed ToE.
So to answer the question: Well, i think if you come with relational database experience which is n-dim - learning string theory first is ... not that stupid. Maybe encouraging people to try this route would be better than forcing them to take the whole curriculum.
Taking discreet numbers as common ground GA provides a stable scaffolding - i think in terms of a finite state machine getting stuff on a screen it's fair to say it's the right tool for that job in this constrained environment.
I also dabbled with scattering amplitudes, but from what i understand so far it's similar to what category theory is in math: Structure before even agreeing on (countable, etc.) sets.
I'm always open for dialog on these and like digging to solid grounds, still i think it makes sense to take a look at the environment then agree on a common ground to build from.
In medicine this can be biochemistry and in computer aided rendering i think OPs "first" principles are not that far off.
> learning string theory first is ... not that stupid
For a different look maybe check out Wolfram's Metamathematics, since it's arguably a candidate for a ToE. Or think about learning math by stating at ZF set theory. I think you might think this is fine at the beginning but are going to quickly hit a wall.
And remember that GA also has lots of limitations. Don't forget that just because you're advancing doesn't mean you've gotten to the beginning.
I wrote an n-dim engine based on graphs -> 4D tree -> rendering you might like as it would basically support any ruleset and honestly after getting there orientation personal for me died down so maybe me talking without not much academic background is a way to find further pointers.
The choice of words for "first principles" is misleading, granted, but the discussion about what these are might be less fruitful than encouraging the author to stay at it and keep it constructive criticism.
This blog post is very much "first principles" for the layperson. It's not the level of "first principles" that a trained physicist would expect (where there would be rigorous derivations involving dielectric functions and the like), but it's great for a layperson who wants to understand why the world looks the way it does.
What the "first principles" are for a layperson and a physicist are different, as is the expected level of rigor in the derivations.
> that everyone understands.
I know this is going to sound critical, but part of me wondered if this wasn't a vibe coded project. The visuals are impressive, but you can go ask gemini canvas to make them and it will. Here's an example
https://jsbin.com/webudilaha/edit?html,output
PS: you need to close the banner at the top I think in order to be able to see the roughness slider.
It needs a little TLC but it works. Even if they weren't vibe coded, I mostly wanted to pass on they probably can be, for anyone who might find that useful for making more articles like this. Even if they just get you started.
Still, cool article!
- 3d primitives (arrows, hemisphere)
- billboarded text rendering
- mouse interaction/canvas rendering that works for multiple scenes
It might be possible to vibe code but you’ll hit a wall of complexity really quick
It is also interesting to look what you can do if you make your rendering even more physically based. One example is wave_tracer which combines wave optics with path tracing to enable simulating wider range of phenomenons. They have nice theory writeup on their webpage:
I think the idea is supposed to be to go back to the "first principals [that you would have been taught]." Which, yes, sometimes you will learn things that show these early teachings were wrong. Often, though, you can get far more mileage out of the naive models than people want to admit.
To that end, I think used judiciously, this has merit. Rarely does anything go against early teaching. It should lean on the specific assumptions and why they do or do not apply anymore, though.
> you can get far more mileage out of the naive models than people want to admit.
I feel like the biggest problem of first principles is that this is almost never where you start. Rather it is something you work towards. Where it then becomes an iterative process expanding/extrapolating out, then coming back in and refining. Low level knowledge is so beneficial, but also very difficult to obtain. It's even more difficult to see how much room is below you without ever having gone down.
And fully agreed on your reasoning on why these are good phrases to keep in mind. I think I would pile on to the "becomes an iterative process" to highlight that "starting from first principals" is more likely to be where folks start. With advanced solutions expanded to cover more and more details that the initial assumptions could gloss over.
I think this is largely justified, if only because "first principals" meaning "absolute foundation" are almost certainly still being discovered.
Not that it's not a useful phrase—of course it is. But it seems like it's an abuse of what should be called "core agreed assumptions" or something.
Sorry, but no, nerds are huge bikeshedders, and often miss the forest for the trees. I think this also explains how dismissive you are of experts who have actually worked with Musk and don't seem to share your low opinion of his expertise in certain areas. Often the importance of nuance is overblown, as it typically only has third or fourth order effects that can be ignored in favour of focusing on the first and second order factors that dominate.
> nerds are huge bikeshedders
Think about it this way, every big problem can be broken down into a bunch of much smaller problems, right? That's generally how we solve things. So obviously small problems come together to create big problems. The main difference between an expert and a novice is the ability to see how these small things interact. So, how do you, as a non-expert, know if the nuance is unimportant or important?
> how dismissive you are of experts who have actually worked with Musk and don't seem to share your low opinion of his expertise in certain areas.
[0] https://www.businessinsider.com/elon-musk-twitter-fire-staff...
First and second order effects dominate almost everything. Always insisting on nuance therefore is statistically more likely to be bikeshedding than not.
> Experts currently working with him or experts who used to work with him?
Both, and also experts who haven't worked with him in an official capacity but have had extended conversations with him. I have zero interest in retracing all of the old arguments about Musk's competence, save to say that those who dismiss him seem to exhibit just as much deluded confirmation bias about his competence as his fanboys.
It is difficult to get a man to understand something, when his salary depends on his not understanding it.
---
I think there's also something of a "gell-mann amnesia" effect going on here. I could buy him being a manufacturing genius or whatever for a while because I know nothing about car or rocket manufacturing. But as soon as he bought twitter, I realized he was an absolute moron who had never built or successfully software himself in his life (or perhaps his brain had simply severely atrophied since his time at PayPal to the point he should be aware), but wanted to sling buzzwords as if he were a coder.
So making one bad investment, whatever his reasons, means he's a moron?
> but wanted to sling buzzwords as if he were a coder.
Yes, he has a habit of trying to build excitement around his endeavours, and part of that is intentionally projecting an image of technical competence because that's his "brand".
I would agree that software is not one of his strong suits, probably because it was never a specific interest of his, unlike electric vehicles and rockets. Is it your opinion that Musk is not aware of his lack of technical competence with software, or are you saying that his lack of technical competence in this domain combined with the fact that he intentionally tried to project competence means he's also not competent in those other domains?
Good use of Gell-Mann Amnesia too. I have started using it as a litmus test of sorts. When I encounter a new source I'll go look for something I have domain expertise in. If it seems accurate enough, I'll tend to trust domains I don't have expertise in. If it is inaccurate, I just don't trust them. Actually this is also a strategy I suggest people use with chatbots, as sometimes small details can be critical while other times they are inconsequential. Since the chatbots are not great at nuances this tends to be a good check, but the difficulty is ensuring you prompt as naively as you would in a subject you're less knowledgeable in.
But no, roads are not designed with just vision in mind. Designers use tecture not just for grip but to help communicate things to the driver. There's many subtler ones, but the most obvious one is the grooves you often find on the edge of highways that are used to warn you if you're veering off. This vibrates the car and creates a loud noise. That's two more senses that you're constantly using while driving even if you don't recognize it. Sure, I wouldn't rely on smell, but it is also a useful sense for some diagnostics and may help in some edge cases. But my point is that we're not just vision based creatures. You think about vision more, but the others are very important.
This is a red herring. None of these feedback mechanisms existed for decades of driving. The core problem of driving is vision and everything else is just gravy. If lidar can't solve all of the vision issues, which it can't, then it makes perfect sense to ask whether vision can cover lidar's purposes and thus whether having both is actually useful. Focusing on lidar is ignoring the core issue.
> This is a red herring. None of these feedback mechanisms existed for decades of driving
Are you claiming humans didn't have the sense of sound nor the sense of touch until relatively recently?
Or are you claiming that as soon as someone enters a car these senses go away?
Are you arguing you can't hear things while in a car? Windows up? Windows down? In an open car like a convertible, jeep, or a model T?
Are you arguing that you don't feel bumps in the road?
Are you arguing you can't tell the difference between driving on asphalt vs concrete?
I think we're done here because you need to get an EKG as soon as possible.
My claim was pretty obviously about intentional multisensory feedback design of roads.
> I think we're done here because you need to get an EKG as soon as possible.
You're right, we are done because you seemingly can't understand the incredibly simple point that you can drive if you lack all other senses except sight, but you cannot drive if you have all of your other senses except sight, and that this means something pretty critical about the importance of sight over all other senses.
If so, do you think he's stupid? Do you think he thinks sensor fusion is a grand unsolved problem his cracked engineers can't implement?
I absolutely agree vision is very very important, given the roads + signage are designed for humans using their eyes among other factors.
Do I think we should take the abilities of human drivers with eyes as the gold standard for what we can achieve with autonomous vehicles? No, and I doubt he does either, but his ego can't backtrack. In fact, I'm pretty sure he finally accepted radar.
Do I believe that looking at humans driving and saying "they do it just their eyes (they don't) so we should!" is an example of his acclaimed first-principles reasoning ability? No, I think its shallow reasoning by analogy.
Also, I'm not aware of anyone going or championing full lidar. It was only musk who (apparently) believes more than one type of sensor is a negative.
What really gets me is that musk, the man who's proven just how good he is at scaling & supply chain, somehow didn't think lidar wouldn't dramatically come down in cost.
My most charitable interpretation is that he really saw the value in gathering as much data about driving as fast as possible, and the best way to do that was to build a bunch of cars with just cameras.(Smart) Somewhere along the way he got annoyed or jealous of the lidar-incorporating companies that were doing quite well with autonomy. Then some ego thing happened and he got ahead of his skis and said vision-only is the best(dumb) and his ego/brand value won't let him take it back.
I admire the hell out of musk based on his output, so I choose to believe that he can't possibly be stupid enough to not understand sensor fusion.
Oh back to the original subject. If he's never said that vision-only is first principles, I apologize for the misinformation. But you have to admit that the "reasoning" of saying humans can drive with vision only therefore no other sensors is quite firmly outside the realm of first principles, which is his m.o.
> Do I believe that looking at humans driving and saying "they do it just their eyes (they don't) so we should!" is an example of his acclaimed first-principles reasoning ability? No, I think its shallow reasoning by analogy.
Elon's understanding of "first principles" is "the first thought that came into my head". Unfortunately this is a common, and growing, bastardization of the phrase.
Lol, I would be a little more cynical and say "the first thought that comes to my head that justifies my business decisions (which I can never show any ounce of doubt over)"
Maybe it can't be solved using current tech, who knows, but saying that fully autonomous driving in our current world isn't achievable without vision is a correct position. I have no opinion on anything else.
> see human drivers using only vision
Anyways thanks for the feedback I’ll look into it
FF: 142.0.1
OS: Android 15
Beaware though, it's a 8000+ lines of code js file that is very badly organized, it's by no mean a reference for good quality code. However I find writing everything by hand easier in the long term than using already existing libraries for example.
The code includes all the math functions, mesh primitive generation, shaders and even a sketchy text renderer using sdf fonts.
If I had to make it again I would use typescript, type errors were the biggest source of bugs.
I'm a big fan of zero-dependency code (or at the very least with any dependency vendored/hosted locally), it means this page will still work and look as it does today in 25 years time.
I don't know if TS runs natively in browsers yet, but v8 / NodeJS does support it (just strips off Typescript specific tokens).
The term subsurface scattering is used for light that exits further away (more than a pixel) like in simulations of human skin
However that part of the article isn’t clear enough and warrants a rewrite
Small nit on the wording: "This continuous cycle enables self-sustaining, self-propagating electromagnetic waves"
The term “self-propagating” makes it sound like the fields are somehow pushing each other forward, but really they’re just coupled solutions of Maxwell’s equations.
Can't we measure micro-facet depth and angle distribution of real world materials using laser speckle and use it to inform the normal distribution function? Deriving ideals from first principles is great and all, but verifying them against real world measurements seems like the bare minimum.
https://www.researchgate.net/figure/Picture-of-the-BRDF-meas...
https://www.iosb.fraunhofer.de/en/projects-and-products/sig_...
Real life measures are also useful for inverse rendering.
It looks like a good model for lower incident angles but begins to diverse at higher angles. There's also one huge caveat - it relates the GGX model to real world ground glass. Obviously it's important to compare like to like. However, very few materials in the real world are ground glass.
Looking around my surroundings, I see painted drywall, wood, paper, plastics, and textiles. It would seemingly be a mistake to assume each of these materials behaved optically like ground glass. I hope that similar measurements are carried out on other materials.
But I agree there should be a lot more work in this area!!