mochomocha

Seeing the GOP trying to repeal the IRS funding boost as soon as they have majority in the House, it's hard not to become cynical about how corrupt our politicians are. The latest CBO report [1] estimates that extra IRS funding has an ROI of 5 to 9.

[1]: https://www.cbo.gov/publication/57444

The number of companies ready to spend $40M/year of compute per "average ads model engineer" is exactly 0.

I think the author downplays the significance his work because it only applies to "small neural networks". There are a lot of use-cases that can benefit from this type of optimizations. Discovering how to use an undocumented fast accelerator available on millions of devices is very valuable.

The analogy doesn't have to hold on every possible behavior of the two entities compared for OP to have a valid point.

If you're going to correct someone, you'd likely want to do it right: he did not want to spend that kind of money after having agreed to do so in a legally binding way. Mark of a business genius who is definitely not impulsive and driven by ego.

It's definitely not an idiotic approach, and companies do keep buffers. In the most advanced cases, probabilistic models are devised to estimate how big these buffers should be. Asking for companies to keep buffers for unpredictable "once in a century"-type events is unrealistic.

Yep. Much harder to be the one bearing the responsibility of the calls being made than being an armchair commentator complaining about the decisions. I don't anticipate that he'll be any more successful at this than other leadership teams of social media companies. There's no way of "winning" at this game: damned if you do, damned if you don't.

The biggest surprise to me from this article was learning that Levandowski is not in jail anymore and back at it.

The whole article reads like free publicity for him and his new company, and is light on data backing up the claims (beyond usual anecdotes).

In a soon-approaching world where all movies have deep-fake actors, popular music is generated etc. how do you approach the economics of creativity and content generation?

Should Tom Cruise heirs receive a perpetual rent 200 years from now when Mission Impossible 57 staring their ancestor is airing?

What regulation should be put in place / would be effective in a world where any teen with the latest trending scoial media app on their phone can realistically impersonate a celebrity in real-time for likes?

Yes, but I don't think SF is an exception here. The whole Bay Area is a NIMBY nest that looks frozen in time with infrastructure similar to what was in Europe in the 1950s. That's how my SF neighbor pays 30x less property tax than I do though, so there's that.

Too much local democracy: "I don't want someone in Sacramento deciding for me" is how you end up with this expression of pure individualism, where Atherton residents who have never taken public transportation in their life can block Caltrain development used by the working mass.

I have a non-pro version, but I can definitely relate to the "too slick, it randomly slides" problem - especially bad because the back camera protrudes. It's pretty ridiculous, my phone ends up on the floor multiple times a day despite me being the careful kind when it comes to electronics.

Subnormals in particular I think are a pretty big failure of the IEEE standard, in the sense that because their implementation is very slow & varies across architectures, the only time most people learn about them is when they're hitting FP perf issues & want to disable their support.

I think the theory you're describing is pretty interesting, I had never thought about depression this way.

Categorical variables in the datasets of large tech companies can take a lot of different values.

I've put in production numerous models with millions of tabular data points and a 10^5-10^6 feature space where tree-based models (or FF nets) outperform more complex DL approaches.

> Why Should I Pay for Peak Load? [...] someone in that supply chain is charging you based on their peak load

Oh it's even worse than that: this someone oversubscribe your hardware a little during your peak and a lot during your trough, padding their great margins at the expense of extra cache misses/perf degradation of your software that most of the time you won't notice if they do their job well.

This is one of the reasons why large companies such as my employer (Netflix) are able to invest into their own compute platforms to reclaim some of these gains back, so that any oversubscription & collocation gains materialize into a lower cloud bill - instead of having your spare CPU cycles be funneled to a random co-tenant customer of your cloud provider, the latter capturing the extra value.

As someone eagerly waiting for his Prophet5 Rev4 to arrive by mail by the end of the week, this reassures me :) Been playing on software versions of the Prophet5 for years and I finally made the jump and bought "the real thing".

Amazing how fast the US is turning into a populist autocracy.

Given that cloud providers oversubscribe their rack power supplies for $ reasons, I'm waiting for the cloud-level equivalent of this DVFS attack, where you throttle a competitor cloud instances by bursting on collocated instances of yours :)

A friend of mine didn't get a promotion at Google because he was told that though his work generated >1B of revenues for the company, it was not "hard enough". He left the company.

Can you expand on why you find AVX512 instructions more innovative? I haven't had a chance to try SVE yet, but on paper it sounds very innovative and offers a wide range of new capabilities.

Gather/scatter have been around for a while, but it hasn't been until more recent Intel uarch that their cost makes them worth using in practice. Zen3 is still lagging quite a bit.

Right, all this overhead is very pytorch specific and doesn't have to be present in all cases. If you resolve everything at compile-time for a limited class of cases (which is what vectorflow does), it's possible to have a solution where none of this overhead exists at runtime (no dispatch, no indirections, no memory copies nor allocations...), neither during training nor serving.

I think pytorch has a genius design and makes excellent tradeoffs. But it's not the perfect solution to every problem. Genericity vs specialization: hard to win on all fronts.

If you're loading the model for every prediction, you're doing something really wrong. Pre-processing and post-processing are indeed usually trumping the core computation itself, but as you mentioned it's not done in the NN.

It's not developed anymore. I still have a service in production using it: estimating the distribution of runtime of any container we launch, through joint conditional quantile regression (output of the net is p5, p10, ..., p95 of the runtime distribution).

It's a good use case for it, because it needs to be low-latency (it's in the critical path when launching a container at Netflix) and it's using a lot of sparse features to do its job (hashed tokens of a lot of poorly structured metadata about the container and its runtime).

Would you mind defining "overhead"?

A library I designed a few years ago (https://github.com/Netflix/vectorflow) is also much faster than pytorch/tensorflow in these cases.

In "small" or "very sparse" setups, you're memory bound, not compute bound. TF and Pytorch are bad at that because they assume memory movements are worth it and do very little in-place operations.

Different tools for different jobs.

Right, it's a very slow bandaid. If you're reaching subnormal numbers, it's usually a sign you're doing something wrong.

Yep. The other advantages are that now you can parallelize compute, and you get lower latency.

Very cool work, congratulations on the launch! Can you comment on how you see the trend of edge computing evolve in the future for SBCs? In terms of perf per watt, could FPGAs compete against a coral-style TPU? What if we had open Mali GPU or NPU APIs to program against the chips already present on SBCs? I'm just a hobbyist so I know very little of what people actually deploy in industrial settings - which would be your target customers.

I'm curious if you have any benchmark (or anecdotal evidence) on the relative perf&power efficiency of using the DSP blocks of the FPGA boards or not?