I think you’re overthinking this, e.g., Zemax’s optimization isn’t that different than the ray-tracing presented in this book. The sophistication truly comes from the users.
You’d be surprised! Everywhere I’ve worked, academic or industry, typically writes their own simulation software. Sometimes it’s entirely handwritten (i.e., end-to-end, preprocessing to simulation to evaluation), sometimes it’ll leverage a pre-existing open source package. I imagine this will become more and more common if, for no other reason, you can’t back-propagate an OpticStudio project and open source automatic differentiation packages are unbeatable.
I used to do this and, for me, the time it took was entirely dependent on the gear I was provided and whether the climb was enjoyable. The gear isn’t exactly what you think of when you think climbing gear and, frankly, it’s infuriating. It might be manufactured by Petzl but the weight and utility are terrible relative to their alpine and mountaineering products. Hell, I was expected to wear work boots that were extremely heavy and climbed poorly. As far as scenery, everyone, myself included, would treat these climbs as climbs and would enjoy ourselves if the views were scenic. The work itself wasn’t hard relative to the trips we were guiding during the climbing season.
> “If you have to have your 15 minutes of fame for something, why not let it be something as joyful as the ‘I Voted’ sticker,” Boudreau says. “It just makes me smile.”
Cargo is great so I’m surprised we’re not seeing tinier interoperable packages (e.g., tasks like serialization and transport) instead of C++-like frameworks.
I worked on Hack during my Meta bootcamp and was surprised to learn that many people used and relied on Hack outside of Meta. It didn’t appear to be anywhere near the size and scope of PyTorch and React but the number of users looked relatively large!
I don’t think it’s more complicated than people want an abstraction for the complex number instructions you find on processors like the TMS320 series or Qualcomm Hexagon.
Apple and Microsoft underinvest in their widget toolkits. It’s especially damning for Apple since they spend the money to write numerous custom widgets (e.g., the widgets in their professional apps like Final Cut Pro and Logic Pro) but don’t upstream. Nobody is inspired by yet another implementation of button or tab widgets.
Unless you know something I don’t, that’s not the case. It also makes sense, engineers are far more portable and scarcity isn’t an issue (many ML PhDs find engineering positions).
Academic compensation is different than what you’d find elsewhere on Hacker News. Likewise, academic performance is evaluated differently than what you’d expect as a software engineer. Ultimately, everyone cares about scientific impact so academic compensation relies on name and recognition far more than money. Personally, I care about the performance of the researchers (i.e., their publications), the institution’s larger research program (and their resources), the institution’s commitment to my research (e.g., fellowships and tenure). I want to do science for my entire career so I prioritize longevity rather than a quick buck.
I’ll add, the lack of compute resources was a far worse problem early in the deep learning research boom, but the market has adjusted and most researchers are able to be productive with existing compute infrastructure.
Hacker News is, in my experience, and despite its benefits, susceptible to a belief in pseudoscience. I love the optimism behind the refrain of “what’s next?” But, at the end of the day, I wish the community, myself included, did a better job of acknowledging that this optimism too frequently sacrifices comprehension and understanding for the dopamine released when we contemplate possible futures.
There’re a handful of ways to introduce electro-optical and all-optical non-linearities into optical neural networks. Here’s a fair survey of the field: https://dl.acm.org/doi/full/10.1145/3607533
If you have a laser, you can print an optical neural network using a consumer 3D printer. The present barrier to adoption, as the article describes, is the cost of transitioning from the optical space to the electronic space.
I don’t hate this! The author is, despite their sarcasm, correct—sometimes integers are insufficiently expressive. It’s the C++ equivalent of geometry in the Wolfram Language.
I believe “world models” are the future of the field so I really need better performance in areas like IVP, FFT, special functions (e.g., harmonics), and dynamic programming. The H100 advances (e.g., DPX instructions) are terrific but they feel like a starting point. Hell, improved geometric operations (e.g., triangulation and intersection) would be killer too and surely that expertise exists at NVIDIA! The H100, especially for the price, feels terrible when you’re training a neural network bottlenecked on an operation that flies on a consumer CPU when you know there’s GPU optimizations that have been left on the floor.