In cities where no such facilities exist (public toilets of any kind), you better not walk in quiet alleys off the busy streets if you have a sensitive nose...
276 karma · joined October 16, 2024
I would imagine you're going to step through time anyways, to draw frame after frame. Wouldn't it in fact be cheaper to step through using some finite differences scheme, with one FD step for one draw step? It could be as cheap as a handful of additions per time step. Unless I'm overlooking something?
If needed for stability or accuracy, you can also step through several FD steps in one frame, but I'd imagine at a rate of 20fps, this should not be necessary for the types of equations considered.
https://parishistoryofourstreets.com/2021/03/22/the-last-pub...
This conflation of everything with AI is precisely why people say things like "gradient descent is most often used in ML" without evidence, and this likely being wrong. No, instead it is 1) ML is the currently most prominent (to the public) use of mathematical optimization and 2) everything else is called AI to the public so they conflate that with ML even when it isn't.
Take a random employee at an engineering or applied sciences (non experimental) lab, ask them if they ever use mathematical optimization, chances are a majority will tell you they do. The vast majority of these are not using or devising ML algorithms.
This matters because of what is clear from this thread. Some people devise a classic algorithm that requires intimate knowledge of the problem at hand, the press calls it AI, the public thinks it's AI, registers one more case of "AI as the tool to replace all others". The Zeitgeist becomes that everything else can go to the bin, and AI (by the more restrictive definition) receives disproportionate attention and funds. Note that funding AI research would not fund the people in the headline, unless they do like the minority of bandits that rebrand their non-AI work with AI keywords.
Another way to look at it, is only those who could write very optimized code could get anything meaningful running on those machines. Hence all (?) software from far enough back was optimized down to death.
I'm personally grateful I've never had to wrangle hardware bugs into features and other such hacks one stumbles upon in anecdotes from those eras... even though there is a certain charm to it all.
People were already using those computers for applications that go slightly beyond animating 5 x 3 green little bars up and down at the time...
I understand dev time cost > CPU time cost (for the company that hires the developpers anyways) but aren't things getting a little out of hand?
Even without comparing to an 80's supercomputer, what if someone with a 15 years old laptop tries to use their app? What will those 6% CPU on the dev's shiny M2 MBP become?
https://journals.aps.org/prx/abstract/10.1103/PhysRevX.15.02...
I've had a few of those, I think they're usually a symptom of stack corruption.
It is a simple iterative algorithm that goes from one point to the next. It doesn't even have memory of previous steps (caveat, the authors used BFGS which approximates the Hessian with previous gradient iterates, but this is still not AI). There is no finding weights or any such thing.
If every for loop is AI, then we might as well call everything AI. Can you pass me the AI, please?
Why would that be the case, though? It's conceivable that optimal shapes be very different from what our intuitions suggest.
- methods that were devised with domain knowledge (= numerical methods)
- generic methods that rely on numerical brute forcing to interpolate general behaviour (= AI)
The qualitative leap is that numerical brute forcing is at a stage where it can be applied to useful enough generic models.
There's a fundamental difference between any ML based method and, say, classic optimization. Let's take a simple gradient descent. This solves a very specific (if general) class of problems: min_x f(x) where f is differentiable. Since f is differentiable, someone had the (straightforward) idea of using its gradient to figure out where to go. The gradient is the direction of greatest ascent, so -grad(f) comes as a good guess of where to go to decrease f. But this is local information, only valid at (or rather in the vicinity of) a point. Hence, short of improving the descent direction (which other methods do, like quasi-Newton methods, which allow a "larger vicinity" of descent direction pertinence), the best you can do is iterate along x - h grad(f) at various h and find one that is optimal in some sense. How this is optimal is all worked out by hand: it should provide sufficient decrease, while still giving you some room for progression (not too low a gradient), in the case of the Wolfe-Armijo rules, for example.
These are all unimportant details, the point is the algorithms are devised by carefully examining the objects at play (here, differentiable functions), and how best to exploit their behaviour. These algorithms are quite specific; some assume the function is twice differentiable, others that it is Lipschitzian and you know the constant, in others you don't know the constant, or the function is convex...
Now in AI, generally speaking, you define a parametric function family (the parameters are called weights) and you fit that family of functions so that it maps inputs to desired ouputs (called training). This is really meta-algorithmics, in a sense. No domain knowledge required to devise an algorithm that solves, say, the heat equation (though it will do so badly) or can reproduce some probability distribution. Under the assumption that your parametric function family is large enough that it can interpolate the behaviour you're looking after, of course. (correct me on this paragraph if I'm wrong)
To summarize, in my (classic numerics trained) mind, classic numerics is devising methods that apply to specific cases and require knowledge of the objects at play, and AI is devising general interpolators that can fit to varied behaviour given enough CPU (or GPU as it were) time.
So, this article is clearly not describing AI as people usually mean it in academia, at least. I'll bet you a $100 the authors of the software they used don't describe it as AI.
It used to be you could learn to program with a cheap old computer a majority of families can afford. It might have run slower, but you still had all the same tooling that's found on a professional's computer.
To use LLMs for coding, you either have to pay a third party for compute power (and access to models), or you have to provide it yourself (and use freely available ones). Both are (and IMO will remain) expensive.
I'm afraid this builds a moat around programming that will make it less accessible as a discipline. Kids won't just tinker they way into a programming career as they used to, if it takes asking for mom's credit card from minute 0.
As for HS + college providing a CS education using LLMs, spare me. They already don't do that when all it takes is a computer room with free software on it. And I'm not advocating for public funds to be diverted to LLM providers either.
IMO the dichotomy should not be deterministic/stochastic, but proved/unproved reliable. gcc has been shown reliable, for instance, so I don't need to know whether it was built by deterministic (clever engineers) or stochastic (typewriting monkeys) processes. I'm certain the former are more efficient, but this is ultimately not what makes the tool valuable.
As a bit of an artificial example, there's stochastic processes that can be proved to converge to a desired result (say, a stochastic gradient descent, or Monte-Carlo integration), in the same way that deterministic methods can (say a classic gradient descent or quadrature rules).
In practical cases, the only proof that matters is empirical. I write (deterministic) mathematical algorithms for a living, yet they very rarely come out correct on first iteration. The fact there is a mathematical proof that a certain algorithm yields certain results lets me arrive at a working program much faster than if I left it to typewriting monkeys, but it is ultimately not what guarantees a valid program. I could just as well, given enough time, let a random text file generator write the programs, and do the same testing I do currently, it would just be very inefficient (an understatement).
In my opinion, a breakthrough is not the production of new knowledge, it is rather its adoption by the public (beginning with industry).
As such, the rate at which breakthroughs can emerge is bounded by factors external to the producers of breakthroughs. And these outside factors are possibly already limiting.
Another point I would make is that what constitutes a breakthrough is not conditioned by how significant it is, only that it is adopted as a change of processes or mental model. As such, more powerful tools can lead to larger leaps between breakthroughs, but not so much higher rate of breakthrough.
As tools become powerful enough to produce yesterday's year's worth of breakthroughs in a month, then the general public and industry will still wait a year before adopting new technology, only it will see larger progress from the previous iteration. This is in fact the case with LLMs. Even on an avant-garde forum as HN, a very common opinion is "I'm waiting out stagnation before I adopt".
As an over simplification, consider only breakthroughs those that come to have widespread commercial application. If we had an oracle for breakthroughs that could produce arbitrarily many today's-breakthroughs as fast as desired, we'd still be limited by our ability to put them in practice. Work must be allocated, carried out over time, and each new breakthrough requires changing processes and the people involved learning new things, which takes time and energy.
I think this human resistance to change is fundamentally what determines the achievable rate of breakthroughs. As the name implies, a breakthrough is a rupture. It is highly inefficient to be upending one's methods every month. It can even be outright impossible to keep up with all the theoretical advancements, before they have crystallized and been digested into accessible vulgarization, if that is not one's profession (i.e. all time devoted to it).
In my applied sciences field, industry is lagging behind some 20 years. And we ourselves are perhaps a century late to some theoretical advances (I can think of one off the top of my head). At the lowest level, there is resistance to change in that ideas take much longer to be carried to a working prototype, than it takes to have them. Hence, someone who constantly hops to new ideas is guaranteed not to make any progress. By necessity, some stubbornness is selected for. Once things are fleshed out (a multi year endeavour), you still have to convince the broader community (same sub field but not direct collaborators) that your idea has merits surpassing theirs, which is a problem best solved one retirement, and one past mentee hire, at a time. And ultimately convince industrial actors that they should dump millions industrializing these novel methods, when none of their competitors have been doing it (hence it is urgent to wait), the viability (robustness, scalability) of the idea remains to be seen, and the benefits weighed against the risk their practitioner user base won't be able to understand the full scope of the progress and see the need to invest time in learning new things and devising new processes (all of which takes time, money, and makes you dependent on this pioneering supplier). And, lastly, there are three other approaches claiming to be better alternatives.
I don't see a way around this pipeline, and more powerful tools can indeed accelerate some of the stages, but there will remain incompressible delays. Ideas need time to be diffused and understood, all the more if they were advancing at a rapid pace enabled by powerful AIs.
Another example is the eigenvectors of linear operators like the Laplacian. Recall how, in finite dimension, the eigenvectors of a full rank operator (matrix) form an orthonormal basis of the vector space. There is a similar notion in infinite dimension. I can't find an English page that covers this very well, but there's a couple of paragraphs in the Spectral Theorem page (https://en.wikipedia.org/wiki/Spectral_theorem#Unbounded_sel... ). The article linked here also touches on this.
Regarding your last sentence, one thing to note is that having a basis is not what makes you a Hilbert space, but rather having an inner product! In fact, to get the Fourier coefficients, you need to use that inner product.
Basically, define a finite dimensional function space V_N of dimension N, in such a way that you could grow N to be arbitrarily large. Solve not the PDE (originally defined over an infinite dimensional function space V, such as H^1), but its discretization as if it dealt only with functions in V_N rather than all functions. The PDE is then simply a linear system, easy to solve. And you can prove, for instance in the case of elliptic PDEs, that the solution to the discrete problem is the orthogonal projection of the true solution of the PDE (in V) onto V_N (Céa's Lemma). Finally, you can produce estimations of the error this projection incurs as a function of N, and thus give theoretical guarantees that the algorithm converges to the true solution as N goes to infinity.
(N in this case is the number of vertices in a mesh that is used to define the basis functions of V_N)
Basically it's an introduction to "functional analysis", the field of math that looks at functions like vectors. The term is in there sneakily, but this is really what the article is all about.
I had classes on this subject and we were encouraged to build upon linear algebra intuition (much like the author does), with some caveats (such as a bounded closed set is not necessarily compact in infinite dimension). Things break down, as the author hints at when they mention they didn't prove the Laplacian to be self-adjoint but only symmetric (this has to do with the operator's domain, a concern that doesn't exist for linear operators on finite dimensional spaces)... but it's still a very convenient way to think and talk about things, conceptually.
Note if you can fit a BREP, it's fairly trivial to find whether a curve is close enough to a straight line that you can just stipulate it's a straight line (same for a plane).
Have you looked into NURBS fitting through point clouds? I understand those can be noisy and over sampled. A colleague got away with sorting point clouds by a Hilbert curve (or other space filling curve) and then keeping 1/N points (just by index), a simple but elegant way to remove N-1 every N points while keeping the general distribution mostly intact (you could also use an octree). Though I recall in some cases the distribution of points was not uniformly too dense, but e.g. dense along scanning lines and sparse between those lines.
Once it's tractable to triangulate the point cloud, you have two important pieces of information at your disposal: local connectivity (prior to that, it'd have been nlog(n) at best to find nearby points) and a notion of topology after some basic processing (e.g. detecting ridges to make out surface patches). With the former, you could do things like smooth the surface to do away with noisiness (say your points are randomly a small distance away from the plane, but any shape really) for better NURBS fitting, estimating normals, etc. and with the latter you could split the domain into faces and curves for your BREP.
At the very least, you'd get much cleaner data to feed an ML algo than basic point clouds. I just find it strange to tackle the raw data head on when there's so many methods for dealing with geometry already, at the very least to clean things up and make some sense of the data.
Have you looked at existing products that do point cloud -> CAD? What are they lacking in?
Regarding what you say of planes and compression, you can look into metric-based surface remeshing. Essentially, you estimate surface curvature (second derivatives) and use that to distort length computations, remeshing your surface to length one in that distorted space, which then yields optimal DoFs to surface approximation error. A plane (or straight line) has 0 curvature so lengths are infinite along it (hence final DoFs there minimal). There's software to do that already, thought I'm not sure it's robust to your usecase, because they've been developed for scientific computing with meshes generated from CAD (presumably smoother than your point cloud meshes).
I'd be really curious to know more about the type of workflow you're interested in, i.e. what does your input look like (do you use some open data sets as well?) and what you hope for in the end (mesh, CAD).
In practice, can you use any of these models with existing code bases of, say, 50k LoC?