100x defect tolerance: How we solved the yield problem

>It's very different from crypto, the main use case for which appears to be money laundering.

Which has substantial economic value (for certain groups of people).

> The current AI tech has opened up paths to develop applications that were impossible just a few years ago.

My argument is that if it's advertised as a direct precursor to AGI based on wishful thinking and people don't know any better, then it's no different to claims about how putting blockchain technology in X industry will solve all of its problems.

I use LLMs daily and don't scoff at AI generated imagery or use cases like agentic systems, but there absolutely is a similar hype cycle to every other innovation out there where people are borderline delusional in the initial stages (Kubernetes will solve all of our issues, moving to cloud and microservices will solve all of our issues, the blockchain will...), before the limitations crystallize and we know what each technology is good or bad at.

Though maybe that says more about human nature than the technology itself.

> It's very different from crypto, the main use case for which appears to be money laundering.

That's akin to saying "The main use case for AI appears to be stealing people's art and even for writers and others it seems to be firing people to replace them with soulless AI generated slop."

I'd even argue that there's nothing wrong with the technologies themselves, be it LLMs, AI for image, video, audio generation, blockchain and crypto, or whatever. The problems arise based on how the technologies are used, or in my argument above - how they're touted as the solution to all the problems. Some people profit a lot, others collide with reality and their limitations at speed.

In other words, if the technology will generate 100 billion USD of actual value but people are betting on 500 billion USD, then clearly we have a bit of an issue.

>the main use case for which appears to be money laundering.

You say tomato, I say freedom from the tyranny of fiat power structures.

> Even if the tech freezes in place, I think it will yield substantial economic value in the coming years.

The question is, where will this "economic value" be? Because "economic value" and actual progress that helps society are two very different things. For example, if someone wants to hire people, they can use "AI" to sift through the applications. But people looking for a job can also use "AI" to write their applications. In the end you may have created "economic value", but its an arms race and a waste of resources at the core, more digital paperwork, a waste of compute. So the actual value is not positive, it is negative. And we see that in many places where this so called "AI" is supposed to help.

Does it mean it is entirely useless? No, but the field of applications where it actually makes sense and has an overall net benefit is way smaller than many believe.

Plus, there are different types of neural networks in use for decades already. Look at OCR, for example, where the commercial OCR software switched to neural networks around the mid 90s already. So it is not that "AI" as such is bad, just that this generative neural network stuff is overly hyped by people who have absolutely no clue about it, but have to hop on the bandwagon to not be left out and keep shareholder values up, because most of these shareholders are equally stupid about the whole issue. Its a circus that burns many resources, money that could have created way more value in other areas.

> Both Kubernetes and serverless (FaaS) is here to stay.

*in environments and projects where they are a good fit

> Microservices is just an excuse to build shit software.

*in environments and projects where they are a bad fit

It does that all the time even now. Play with temp, top_k, min_p and nothing is preventing you from getting combinations and order of tokens previously not encountered in history.

I did eventually get an LLM to produce what seems to be a correct diagram of a sentence it had never seen, but it took about ten tries. Grammatical analysis seems to have happened correctly every time, but getting to a usable diagram was difficult.

(I know that it's generally rude to include LLM output in HN comments, but in this case I think it's essential supporting material to elevate the discussion of LLM capabilities above "yes it is", "no it isn't".)

First, I just asked ChatGPT (GPT-4o mini, what they offer anonymous free users):

> Please diagram the sentence, "If my mother's road had been cultural, my uncle's sloth squad residue would have compiled a periodic meadow of night."

I just invented that sentence, picking several words with a random number generator, but of course I can't prove to you that I didn't copy it from some book you haven't seen. (For that, you'd have to invent a sentence yourself, or at least provide me with some words to include in it.) I hope you can at least agree that it's a highly unlikely sentence for anyone ever to have uttered before!

After talking to itself for a few paragraphs, it arrived at the following grammatical analysis:

  Step 4: Diagramming the sentence

    Main Clause:
        Subject: "my uncle's sloth squad residue"
        Verb: "would have compiled"
        Direct Object: "a periodic meadow of night"
            "a periodic meadow" (noun phrase)
            "of night" (prepositional phrase modifying "meadow")

    Conditional Clause:
        Subject: "my mother's road"
        Verb: "had been" (linking verb)
        Predicate Adjective: "cultural"

This is exactly correct. From that it produced the following ASCII art "diagram", which looks like nonsense to me, but maybe it's in some kind of sentence-diagramming formalism I haven't seen:

                If
        ______/______
       |              |
    my mother's    had been
      road             |
                        cultural
       
   my uncle's sloth squad residue
                 |
    would have compiled
                 |
  a periodic meadow of night
              |
            of
            night

I told it the diagram looked like nonsense, and it produced this one, which looks worse:

                 If
           _______/______
          |               |
  my mother's road    had been
           |             |
       cultural      (Main Clause)
                       |
       my uncle's sloth squad residue
                           |
                 would have compiled
                           |
               a periodic meadow
                         |
                       of night

I asked GPT-4 (paid) the same question, and it gave me another exactly correct grammatical analysis in words:

  1. **Identify the main clauses**: 
     - "If my mother's road had been cultural" is a dependent clause (specifically, a conditional clause).
     - "my uncle's sloth squad residue would have compiled a periodic meadow of night" is the main clause.

  2. **Start with the main clause**:
     - Subject: "my uncle's sloth squad residue"
       - "my uncle's" modifies "sloth squad," which in turn modifies "residue."
     - Predicate: "would have compiled"
       - This includes the modal auxiliary verb ("would"), perfect auxiliary ("have"), and main verb ("compiled").
     - Direct object: "a periodic meadow of night"
       - “a” is an article modifying “meadow,”
       - “periodic” modifies “meadow,”
       - “of night” is a prepositional phrase modifying “meadow.”

  3. **Diagram the dependent clause**:
     - Subject: "road"
       - Modified by possessive noun phrase “my mother’s”
     - Predicate: "had been"
     - Complement/Adjective: "cultural"

(I feel like this is definitely "a level that you'd assume to be competitive to a human", in mschuster91's phrase; most native English speakers can't do that.)

But then it came up with the following incorrect diagram:

                            +---------------------------------------------------+
                            |                      Main Clause                   |
                            +---------------------------------------------------+
                                    /                |               \
                                   /                 |                \
                                  /                  |                 \
                         [My uncle's]         [sloth squad]          [residue]
                             \                    /                     /
                              \                  /                     /
                               \                /                     /
                                would have compiled -------------------
                                                  \
                                                   \
                                                    \
                                                     [a periodic meadow of night]
                                                      /           |            \
                                                     /            |             \
                                                    a        periodic       [of night]

  +---------------------+
  |  Dependent Clause   |
  +---------------------+
        If
         |
        road
      ___|_____
    |          |
   my mother's had been cultural

Telling it the diagram was nonsense didn't help.

But my conclusion from this is not that GPT-4 can't analyze the grammatical structure of sentences it had never seen. It's that GPT-4 gets "confused" when it tries to draw ASCII art, like a person trying to tell you how to draw it over the telephone, but not quite as bad. I've seen similarly abominable results when it tried to draw circuit schematics and guitar tablatures as ASCII art. I tried asking it for a graphviz diagram, which was syntactically correct graphviz code but didn't express the sentence structure properly.

I tried simplifying the problem with the prompt, 'Please provide a syntactic tree showing the syntactic derivation of the sentence, "My uncle's sloth squad residue would have compiled a periodic meadow of night," using graphviz syntax and with individual words at the leaf nodes of the tree.' which yielded a slightly incorrect diagram. (I'm still sure nobody has ever uttered this sentence before.) On being told, "This diagram is not quite right.", GPT-4 produced the following diagram, which I believe is correct except for an extra "s" on "uncle" (you can see this in your browser at https://is.gd/Axufjx):

  digraph G {
      node [shape=plaintext];

      S [label="S"];
      NP [label="NP"];
      VP [label="VP"];

      S -> NP;
      S -> VP;

      // Noun Phrase breakdown
      DP_my [label="DP"];
          DP_my -> My;

      UnclePossessive [label="NP"];
          UnclePossessive -> Uncles;
          UnclePossessive -> ApostropheS;

          NP -> DP_my;
          NP -> UncleCompound;

      UncleCompound [label="NP"];
          SlothSquadResidue[label = "N'"];
              SlothSquadResidue-> Sloth;
              SlothSquadResidue-> SquadResidue;

              SquadResidue[label = "N'"]
                  SquadResidue-> Squad;
                  SquadResidue-> Residue;

              UncleCompound-> UnclePossessive;
              UncleCompound-> SlothSquadResidue;


      // Verb Phrase breakdown
       ModalWould[label = "Modal"];
       ModalWould-> Would;

       AuxHave[label = "Aux"];
       AuxHave-> Have;

       VCompiled[label = "V"];
       VCompiled-> Compiled;

       ObjectNP[label = "NP"];  // Direct object noun phrase
           ArticleA[label = "D"];
           ArticleA-> A;

           MeadowPhrase[label = "N'"];  // 'Meadow' modified by 'periodic'
               AdjPeriodic[label = "Adj"];
               AdjPeriodic-> Periodic; 

               MeadowWord[label = N];
               MeadowWord -> Meadow; 

               MeadowPhrase -> AdjPeriodic; 
               MeadowPhrase -> MeadowWord; 

           PPofNight[label = PP];  // Prepositional phrase 'of night'
               PrepOf[label=P]
                   PrepOf-> Of; 

                   NightNounPhrase[label=N]
                       NightNounPhrase-> Night; 

                   PPofNight -> PrepOf; 
                   PPofNight -> NightNounPhrase; 

           ObjectNP -> ArticleA; 
           ObjectNP -> MeadowPhrase; 
           ObjectNP -> PPofNight;


       VP -> ModalWould;
       VP -> AuxHave;
       VP -> VCompiled
       VP -> ObjectNP

  }

Claude produced a reasonably serviceable diagram on the first try with that prompt:

  digraph SyntacticTree {
    node [shape=plaintext];

    S -> NP1;
    S -> VP;

    NP1 -> DET1 [label="Det"];
    NP1 -> NP2 [label="NP"];

    NP2 -> NP3 [label="NP"];
    NP2 -> N1 [label="N"];

    NP3 -> N2 [label="N"];
    NP3 -> POS [label="Pos"];

    VP -> AUX1 [label="Aux"];
    VP -> AUX2 [label="Aux"];
    VP -> V [label="V"];
    VP -> NP4 [label="NP"];

    NP4 -> DET2 [label="Det"];
    NP4 -> ADJ [label="Adj"];
    NP4 -> N3 [label="N"];
    NP4 -> PP [label="PP"];

    PP -> P [label="P"];
    PP -> NP5 [label="NP"];

    NP5 -> N4 [label="N"];

    DET1 -> "My";
    N2 -> "uncle";
    POS -> "'s";
    N1 -> "sloth";
    N1 -> "squad";
    N1 -> "residue";
    AUX1 -> "would";
    AUX2 -> "have";
    V -> "compiled";
    DET2 -> "a";
    ADJ -> "periodic";
    N3 -> "meadow";
    P -> "of";
    N4 -> "night";
  }

On being told, I think incorrectly, "This diagram is not quite right.", it produced a worse diagram.

So LLMs didn't perform nearly as well on this task as I thought they would, but they also performed much better than you thought they would.

A language model can generate valid sentences that have never existed before -- not just syntactically or statistically valid sentences in a Markovian sense, but semantically valid ones. How could it possibly do that without an implicit awareness of all aspects of language, including grammar and parts of speech?

That implicit awareness is indistinguishable from knowledge, at least to the extent it yields correct results.

And the fact that it was inferred from examples rather than taught explicitly is indistinguishable from intelligence.

Yes, I sometimes wonder if what we're witnessing in our lifetimes is the next stage of the 'bootstrapping' of life into a more complex form. If we might be the mitochondria contributing our little piece to the cell that comes after.

Define "by definition".

Because this statement really makes no sense. Transformers are perfectly capable (and capable of perfectly) learning mathematical functions, given the necessary working-out space, e.g. for long division or for algebraic manipulation. And they can learn to generalise from their training data very well (although very data-inefficiently). That's their entire strength!

Yet they can get silver medal PhD level competition math scores.

Perhaps your "definition" should be simply that LLMs have temporarily seen limitations in their ability to natively do math unassisted by an external memory, but are exceptionally good at very advanced math when they can compensate for their lossy short-term attention memory...

> Evolution selects for efficiency.

I think this is a poor argument here. From an evolutionary point of view, our brains are optimized to:

- Provide fine-motor control to craft weapons and tools (enhancing adaptibility and enabling us to hunt way outside our weight class)

- Communicate/coordinate effectively in small groups

- Do sensor processing and the above with a low energy budget

Our brains are *not* selected to be minimum-complexity intelligences, and a lot of what our brain does is completely useless for AGI building (motor control, sensor processing, ...).

Furthermore, the cost/complexity (from a evolutionary PoV) is a totally different beast from what complexity means to us.

Just consider flight as an example: A fruitfly is an insanely simple and straightforward beast, but to us, a biochemically fuelled, beating-wing design is still infeasibly complicated. If our approach to flight had been to ape after how nature does it in detail, we likely still would not have planes.

I do agree that todays LLMs still have clear architectural flaws that we need to overcome (online learning being a very glaring one), but, to pick up the flight analogy, we might well have the main wing structure already down, and we won't necessarily have to make those wings beat to get into the air...

What do our brains do that isn't token prediction?

They receive information about photons and air vibrations and control muscles, okay. If a human brain was hooked up the way ChatGPT was, only to text input and output, would that make it not intelligent?

> What gave you the impression I was assuming bad faith?

You said "I would guess you're not asking a serious question here"

It was a direct quote from your original comment

You brought it up...

Without offering any opinion on its merits, if you think justifying this controversial claim is off topic, then so is the claim and you shouldn't have written it.

You said, "I would guess you're not asking a serious question here," which is to say, you were guessing that the question was asked in bad faith. Or, at any rate, you would, if for some reason the question came up, for example in deciding how to answer it. Which is what you were doing. That is to say, you did guess that it was asked in bad faith. Given the minimal amount of evidence available (12 words and a nickname "__Joker") I think it's reasonable to describe that guess as an assumption. Ergo, you were assuming bad faith.

Right. And any particular question people think AIs are bad at also has a comments section of people who have run better crafted prompts that do the job just fine. The consensus is heading more towards "well damn, actually LLMs might be all we need" rather than "LLMs are just a stepping stone" - but either way, that's fine, cuz plenty of more advanced architecture uses are on their way (especially error correction / consistency frameworks).

I dont believe there are any significant academic critiques doubting this. There are a lot of armchair hot takes, and perceptions that this stuff isn't improving up to their expectations, but those are pretty divorced from any rigorous analysis of the field, which is still improving at staggeringly fast rates compared to any other field of research. Aint no wall, folks.

> This goes so far as to have companies or business units share wafers for prototyping runs so as to minimize cost per device

Can this be done in production? Is there a chance that the portion of the wafer cerebras.ai can't fit their giant square in is being used for production of some other companies chips?

Normally yes. But they're using a whole wafer for a single chip! So it's actually a good idea.

I guess the issue is how do you design your routing fabric to work in the edge regions.

Actually I wonder how they are exposing this wafer. Normal chips are exposed in a rectangular batch called a reticle. The reticle mask has repeated patterns across it, and it is then exposed repeatedly across the wafer. So either they have to make a reticle mask the full size of the wafer, which sounds expensive, or they somehow have to precisely align reticle exposures so that the joined edges form valid circuits.

The article shows them using a single maximally sized square portion of a circular wafer.

I think the proposal you're responding to is "just use the whole circular wafer without cutting out a square".

If you want to have nice straight edges to clamp into place, then you only need to shave off four slivers. You can lose a couple percent instead of more than a third.

They already have a notch or flat for alignment, which is much more critical during the lithography process than during soldering.

You just need a sharpie to mark the top.