Comment by com2kid | Hacker News Reader

Comment by com2kid | Hacker News Reader

The comments on resource utilization are one of those things that is only true now that there is a 1:1 correlation between $ and available CPU/Memory, and the pool of available CPU/Memory resources is effectively unlimited.

Embedded is maybe the last refuge of where there are hard resource constraints that cannot be violated. If you only have 64K of RAM, you had better make it work!

IMHO this is why you can end up with embedded code that is easily 10x to 100x more optimized than code running in other environments. It is also, why IMHO, the user experience on Smart Phones doesn't improve linearly with the hardware improvements - see https://en.wikipedia.org/wiki/Wirth%27s_law

This train of thought, along with basically flatlined GPU performance / $, explains also why we are seeing real algorithmic improvements in the ML/AI space. If we were still operating under Moore's law and GPUs and VRAM were 2xing every couple of years, I doubt we'd see all the research efforts going into optimizing training, fine tuning, context windows, inference, and so forth.

TFA author - I have worked on low-cost embedded systems. (Tens/hundreds of megs of RAM, not KBs.) The only case where you don't hoard resources is if the system is so small that it's programmed by a small team since the functionality is severely limited by resource scarcity, so there's no reason to grow the team to deploy more functionality. Above a certain not-so-large size, people will hoard resources in an embedded system - they can't ask for more capacity like they would if it was server-side software, but they sure can avoid giving up whatever resources their code is already using.

Researchers actually have a limited and smallish hardware budget, so academia is likely to come up with cost-saving ideas even when hardware performance grows very quickly. In the industry you can throw more hardware at the problem even if it's not improving (outside embedded/client devices)

As someone who works in embedded systems- I agree. Teams will deploy whole desktops to an embedded system if permitted to expand enough. Architectures will evolve to require hardware that can support the bloated stack.

Even on power constrained robots operating in the deep sea it's a pretty safe bet that some of them are running whole Windows VMs.

> TFA author - I have worked on low-cost embedded systems. (Tens/hundreds of megs of RAM, not KBs.)

I worked with kilobytes, small team, everyone sat together. No one was hoarding resources, because just to ship out the door we had to profile every single function for memory and power usage!

IMHO Hundreds of MB of RAM isn't embedded, it is just "somewhat constrained". :-D Regular tools run just fine, you can use higher level languages and garbage collectors and such, you just have to be a little bit careful about things.

When you drop under a MB, but are still expected to have a fully modern UI, then things get interesting. (I wrote up a blog post about my experiences working in embedded @ https://meanderingthoughts.hashnode.dev/cooperative-multitas...)

> Researchers actually have a limited and smallish hardware budget, so academia is likely to come up with cost-saving ideas even when hardware performance grows very quickly.

Agreed, but I also think it is difficult to determine when forward progress is stymied by too many resources VS too few!

> the pool of available CPU/Memory resources is effectively unlimited.

This isn't true. Every CPU you use is sitting in a physical server somewhere. Every server is sitting in a rack in a building that's sitting on land that used to be nature. It's using electricity that could be used instead for cooling or heating a person or just not generated at all. It's cooled using fresh water that is needed by every living thing and is in short supply in many places.

The idea that compute is unlimited is a horrible myth that cloud companies have sold which has effects and externalities that will be seen as on par with the myth of unlimited fossil fuels.

> The idea that compute is unlimited is a horrible myth that cloud companies have sold which has effects and externalities that will be seen as on par with the myth of unlimited fossil fuels.

Granted, all true, but my point was that for any one given software team at a mid-sized company with a team of a 50 engineers or so, CPU resources are effectively unlimited. Unless they write some N^3 algorithm, the fact is that it is almost always possible to write AWS a bigger check for more CPU to a pretty much inexhaustible extent from any one given customer's persepctive.

GPUs being the very large exception here!

> If you only have 64K of RAM, you had better make it work!

Hah, this brings back fun memories. I worked for a spell at a large company producing smartphone components. One of their primary products was a do-it-all embedded chip that handled many disparate functions, which eventually grew to rather massive proportions; rather than optimizing the resulting bloated software, the company just petitioned their chip supplier for a special part with way more flash memory than a standard part would have. IIRC it had something like 16x the Flash memory compared to the largest public part.

The code quality was awful. People were churning out C++ monstrosities with zero concern for code size. The final product was the result of linking together modules from dozens of teams, none of whom really cared at all about code size; I’m frankly surprised the CPU didn’t fall over from all the excess code it had to run (the RTOS was, at the very least, very good at prioritizing…). But: I’m sure it saved a lot of engineering effort that went into shipping flashy features instead.

Reply on news.ycombinator.com