Hacker News Reader

Comment by StableAlkyne | Hacker News Reader

Local models are not the same as the giant ones in data centers. They're in about the about the same ballpark as running a AAA game on max settings.

Sure, it's more computationally-expensive than running an HTML selection, but it's also not "staggering" by any reasonable stretch.

View on HN

I think “staggering” is a pretty accurate way of putting it if blocking ads is in the same ballpark as running a AAA game on max settings.

If the computational requirement is within the realm of another normal day to day task, and if it cannot generally achieved with another cheaper approach, it would be contradictory to say it is staggering.

I would agree if this was an easily solvable problem with classical approaches. Clearly it is not, since even uBlock is giving up.

At any rate, "staggering" is relative. White-labeled web browsers becoming the most common GUI engine would have made an engineer's eyes water 20 years ago.

Blocking ads should not make my fans spin at full blast.

Yeah, it's pretty awful that Facebook has made it such a demanding task.

"Honey, I want to buy a gamer PC. This one is too slow to block Facebook ads."

Even the data-center-grade models are less intensive than people think. We recently installed a heavy-duty AI server (Gaudi2 cluster with 768GB VRAM) to run larger models up to 400B parameters on, and at full load, its power draw is about equivalent to a consumer stove or clothes dryer.

If you're running thousands of those concentrated into a single data center, your aggregate power draw is going to be huge, but a single server is entirely reasonable for a small business, or even a home user, to operate.

The giant ones in data centers are models that broadly encompass a variety of data for general purpose applications. You can certainly train models specifically for processing DOM content, making them smaller and more efficient.

Reply on news.ycombinator.com