learndeeply

Partially coincidence, but also ICLR submission deadline was yesterday, so now papers can be public.

Grain increases perceived sharpness and decreases gradient banding, so there's more benefits than just nostalgia.

Most people are buying AMD GPUs for gaming or productivity (e.g. blender), not machine learning, and its working great for them. ROCm is currently a joke. Maybe in a few years AMD will care enough to try to participate in the ML hardware space.

This applies for all neural networks. Depending on how much money you're willing to spend, in descending order: DGX (computer with 8 A100s, $150,000), A100 (80GB, $15,000), A6000 ($5000), RTX 3090 ($1000).

Thanks for answering questions!

Are the example given in the blog post considered zero-shot learning?

Was the model trained on the websites in the examples given (e.g. on the Redfin site)?

How much labeled data was used?

This is mentioned directly in the article:

Taichi vs. Numba: As its name indicates, Numba is tailored for Numpy. Numba is recommended if your functions involve vectorization of Numpy arrays. Compared with Numba, Taichi enjoys the following advantages:

Taichi supports multiple data types, including struct, dataclass, quant, and sparse, and allows you to adjust memory layout flexibly. This feature is extremely desirable when a program handles massive amounts of data. However, Numba only performs best when dealing with dense NumPy arrays. Taichi can call different GPU backends for computation, making large-scale parallel programming (such as particle simulation or rendering) as easy as winking. But it would be hard even to imagine writing a renderer in Numba.

UC Berkeley has a $6 billion endowment. It's not impossible, but improbable.

> If you want to actually deploy AI models

On mobile*.

> I often find myself using the TF data pre-processing pipeline even from inside PyTorch

the tf.data pipeline is quite nice, has some neat auto-tuning features.

I don't think so, Colab pro limitations are precisely because they weren't charging by compute unit, so they were over-subscribed.

Race to the bottom implies that they're only competing on price. Here, they're competing on new functionality as well. If DALL-E's outputs were substantially better than Stable Diffusion, more people would use it, even if it cost more.

It's a race to the top. New functionality is added and the model is improved week over week.

Frustratingly it ignores ViT and other image/video transformer models.

> No one knows for certain how language models that are 10x or even 100x larger than current state-of-the-art ones will perform

Read the Deepmind Chinchilla paper, it answers this

Meta comment: It's pretty bold for a blog that started a year a year ago to call themselves a "news agency" and to claim to speak for the Japanese people.

> My unsubstantiated guess is that this is one of the most comprehensively-typed Python codebases out there for its size.

Not important, but FAANG companies have several orders of magnitude more strictly-typed Python than this.

You're confusing the H100, a high-end data-center card with the RTX 4000 series.

Both - the first is for sharding tensors across GPUs, the second is to do an all reduce (e.g. for distributed data parallel to synchronize gradients)

Out of curiosity, is $10/month prohibitively expensive for you or other developers? Copilot is free for open source developers and students.

Didn't Milvus (vector db, wrapper around FAISS) come before Pinecone?

Why do people keep on re-inventing pipelines? There's so many already, all with almost identical syntax. This one is similar to Flyte.

> But a group of N sqlite databases is an N-writer database. And mvsqlite provides the necessary mechanisms to do serializable cross-database transactions without additional overhead.

I'm confused, are these databases planned to be replicated? Or is it expected for the databases to have separate schemas?

Article was posted in 2019, in case other people missed it like I did.

Unless you've demonstrated that FSD follows scaling laws, this comment is pretty meaningless.

This looks really fun, modern incarnation of Google Wave. Made a test room here: https://yboard.lol/lobby?join=hackernews

If anyone is wondering, Mintlify is not open source: https://github.com/mintlify/mintlify/blob/main/server/LICENS...

How is that related to Sheryl stepping down as COO at all?

I don't get it. What in the license prevents users from removing the telemetry? AGPL just means the user needs to open source that change, right?

Edit: To remove telemetry, just call:

   from mitoinstaller.user_install import go_pro; go_pro();

No licensing or payment required, and doesn't violate the license.

When the concrete weathers, will the plastic in the bricks turn into micro-plastic?

Your questions are answered in the first sentence of the article.

> Zstd or Zstandard (RFC 8478, first released in 2015) is a popular modern compression algorithm. It’s smaller (better compression ratio) and faster than the ubiquitous Zlib/Deflate (RFC 1950 and RFC 1951, first released in 1995).

It's a waste if you're not. https://www.eataly.com/us_en/magazine/how-to/leftover-parmes...