mochomocha

It's literally what I did at work last week, which is why I found this submission timely. I'd have to check with my employer if it can be made public. I don't see any reason why not, there's not much to it.

Fun fact: if you don't care about the auto-regressive aspect of NeuralProphet (it's turned off by default), you can implement the core of NeuralProphet/Prophet (piecewise linear trend + Fourier on weekly/daily seasonality) in about 60 LOCs with no other dependency than either torch or numpy+scipy.optimize, and without having to deal with Stan or the very poorly chosen heuristics of neuralprophet.

Another thing that both NeuralProphet and Prophet do extremely wrong by default is uncertainty estimation. The coverage probabilities are way off.

> It would be useful to know if PyTorch runs faster or slower than their methods using their sparsified matrix techniques.

Probably not an easy feat. Sparse support in pytorch is minimal.

Both Lambda firecracker VMs and t2 instances are multi-tenant and oversubscribed.

My Brother printer once said the cartridge was empty and I refused to believe it. After browsing the internet a bit, I found a trick on an obscure forum that worked for me: you put a small piece of tape on the cartridge chip, and miraculously it's not detected as "empty" anymore. Two years after the procedure, I'm still printing on the same cartridge that was supposedly empty.

To expand a bit on what Aaron said, 2022 was a year where the company experienced drastic culture changes: introduction of levels, new VP&director-level leadership with more traditional / "big corp" management styles, push to hire junior engineers, decrease in transparency during decision-making, to name a few... These changes have alienated a lot of people who self-selected for the unique culture the company had, which was truly remarkable.

Between Marc Tessier-Lavigne, operation Varsity Blues and SBF parents, the scandals involving Stanford keep on coming. It's not sending a good signal when it comes to the overall integrity of the institution.

Slightly off topic, but regarding the larger memory footprint of Kata containers, what is your opinion on KSM effectiveness in general for VMs?

JPEG-2000 uses wavelets as a decomposition basis as opposed to DCT which in theory makes it possible to treat the whole image as a single block while ensuring high compression. In practice though tiles are used, I would guess to improve on memory and compute parallelism.

I was doing the same thing at Netflix around the same time as a 20% research project. Training GANs end2end directly in JPEG coeffs space (and then rebuild a JPEG from the generated coeffs using libjpeg to get an image). The pitch was that it not only worked, but you could get fast training by representing each JPEG block as a dense + sparse vector (dense for the low DCT coeffs, sparse for the high ones since they're ~all zeros) and using a neural network library with fast ops on sparse data.

Training on pixels is inefficient. Why have your first layers of CNNs relearn what's already smartly encoded in the JPEG bits in the first place before it's blown into a bloated height x width x 3 float matrix?

Adding keras abstractions on top of pytorch seems like negative value-added to me. It only barely abstracts more the easy/mundane stuff, while creating yet another layer of indirection between ideas and machine code in the current Rube-Goldberg mess that people call modern ML.

Keras has always been "lipstick on a pig for tensorflow". Its value beyond that seems tenuous at best.

In no world would Youtube be even remotely profitable without advertising. Google ran it as a $B+/year loss for years before managing to making it profitable.

Tell me now a believable alternative growth story for Youtube where Google didn't buy it and the service isn't "corporatized" and not losing money.

You can't compare an unprofitable small-scale startup product to what the same product needs to be to make it 1) planet-scale and 2) profitable.

> But instead AMD has just followed Nvidia into making slow, uncompetitive GPUs at high prices.

I'm curious: do you use GPUs? It's hard to take this claim seriously. Yes NVIDIA GPUs are expensive, but I don't think we live in the same matrix when you claim that their GPUs are slow and uncompetitive... Unless you have an alternative in mind, in which case I'm all ears.

They are usually significantly more expensive than United for all SFO<->CDG flights.

While I agree with the beginning of your post, you lost me here:

> The bitter lesson [1] is going to eventually come for all of these. Eventually we'll figure out how to machine-learn the heuristic rather than hard code it.

Inefficiently re-learning over and over patterns that can be more explicitly encoded as smart inductive biases for better sample efficiency is what ML research is.

The "bitter lesson" doesn't mean "throw the towel and your brain and just buy more GPUs". It means that the inductive biases / modeling strategies that win will always be the ones that are more hardware-friendly.

Good! Everyone will know these new institutions are full of rich people instead of full of smart people, so the job market value of the diplomas they issue will be priced as such (~ 0).

No they shouldn't: anything that's "pay for play" is morally bankrupt. It reinforces the stigma that there are different rules for the rich in this country.

Bribery of congressmen (a.k.a lobbying) or university deans (ALDC admissions) are forms of legal corruptions. You might argue that a private institution can do whatever it wants, and I agree. But in this case it should get 0 tax benefits/subsidies, esp. when they have $B+ endowments.

This is a very good summary of my own thoughts regarding the Supreme Court ruling.

The American obsession with race in this specific case translates into focusing on a more superficial problem and polarizing the debate, as opposed to finding an easier solution to the bigger problem of ALDC admissions (namely: ban them), which I assume would be less controversial.

After all this time if Spotify could have found a way to cut off the middlemen, they would have happily done it. From your own quote, you should be mad at Universal & co, not Spotify.

This is certainly a very interesting thought to entertain and your ideas make sense. One thing that makes things harder on the CPU side in this hypothetical scenario is that CPUs tend to execute much more diverse instructions/computations than GPUs. So all the caching & speculation you mention is probably all the more important.

Perf/$ where $ is what it cost _them_ , not $ they're ready to sell to others as a product. Cloud margins in the high two-digit percents are typical, and I'd imagine even higher for very specialized products in high-demand from deep-pocketed customers.

Yep. "confused" is also how most of his regular papers read to me anyway.

I didn't know what "computational functionalism" was, but after reading about it I don't see how it is remotely the "basic hypothesis behind computer science".

It has always been my favorite museum in Paris: if you're into Kandinsky, they have one of the largest collections.

Also, there's IRCAM right next to it where the mix of music nerds with a wide variety of backgrounds produces very interesting and unique research.

> They may exist for a while

I don't think we should be interested in the "asymptotic theoretical limit" of capitalism which is never reached. Let's discuss "capitalism in practice", not "capitalism in theory". If things self-correct in a time frame longer than your own lifespan, do you think it's an ideal scenario? I personally don't.

I don't see the government forcing Google to pay thousands of highly qualified workers to fill in BS jobs. This is a natural consequence of being a very successful business: complacency, empire building, etc all decorellated from economic incentives.

People spend decades with BS jobs in a very capitalist society.

Note that I've never argued that a socialist alternative is any better or that the government isn't filled with BS jobs either.

I just observe that the problem manifests itself regardless.

> Bullshit jobs don’t exist in the free market, as no capitalist would pay for someone to do something useless

And yet if you spend any time in any large corporation, you'll find countless examples of these, including in the tech industry. Don't you have any friend working for Google/Meta/${big tech co}?

Bullshit jobs can and definitely exist with capitalism: any successful and large business is filled with them.

"Bullshit jobs" [1] are a suboptimal and implicit mean to deliver on UBI without superficially changing the rules of capitalism. For exhibit 2, I'll bring in the US military and its economic impact on the midwest.

[1]: https://en.wikipedia.org/wiki/Bullshit_Jobs

Your take might be honest, but it's clearly uninformed. Everyone has been wrong about how ai developed. It's worth giving "The Bitter Lesson" a read [1] if you haven't yet.

[1]: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

> Refusing to return security deposits is absolutely a low income landlord strategy.

Oh I've had this happen to me from a "high income landlord" in San Francisco. Penny pincher lady living in a wealthy neighborhood, repairing herself the leaky roof of the SF building she owns with duct tape. Tried to keep my deposit until, like the comment you're replying to, I threatened to bring her to small claims court. I had the deposit in my mailbox the next day.

Also working in the field and I don't understand why this article is even on the front-page. The author has clearly no idea what they're talking about, it almost reads as a satire.

They behave this way because they are so tight on GPU capacity given incredible demand. Getting a GPU instance on GCP is as miserable of an experience.

Try Lambda Labs, there's usually more availability, and more importantly you get some visibility into currently available capacity.