alain94040

There is a data dependency: until I read the middle index, I can't tell if I want to search the data in the upper section or the lower section. I can speculate and issue reads to both. That would fix the dependency, but create more memory traffic. Is this the right trade-off?

What do you want? Removing branches is not it.

    assign rv1 = registers[rs1];
    assign rv2 = registers[rs2];

Chip designers intuitively visualize interconnected blocks of logic. They draw rectangles with wires between them.

To learn this intuition, writing spaghetti Verilog code that runs in a simulator is counter-productive. It's a common mistake that people coming from a software background make all the time, and that has been discussed to death. So with some appeal to authority, trust us and learn proper hardware design, hopefully it will be more rewarding and will "click" faster.

Are we sure the author is not showing some unconscious bias by looking in others for traits he has (public creativity)?

There is no particular reason why public creativity is linked to good engineers. It's one way to show good engineering, sure. But far from the only way. I could see some other hiring managers arguing the opposite: if you have time to write lightweight tech posts on your blogs, maybe you are not a great engineer who will solve my company's most difficult problems.

That being said, especially for new grads, I strongly recommend showing something, anything beyond "I graduated from this university and here are the classes I took".

A key architectural feature to achieve this is the ability to fit all model parameters inside the on-chip SRAMs of the chiplets to eliminate bandwidth limitations. Doing so is non-trivial as the amount of memory required is very large and growing for modern LLMs

...

On-chip memories such as SRAM have better read latency and read/write energy than external memories such as DDR or HBM but require more silicon per bit. We show this design choice wins in the competition of TCO per performance for serving large generative language models but requires careful consideration with respect to the chiplet die size, chiplet memory capacity and total number of chiplets to balance the fabrication cost and model performance (Sec.3.2.2) We observe that the inter-chiplet communication issues can be effectively mitigated through proper software-hardware co- design leveraging mapping strategies such as tensor and pipeline model parallelism

But when you read further and see what they tried:

We then selected one Arm instruction from each data-processing bucket,testingstores(str),AESinstructions(aese, aesmc), rotate right (ror), bitwise and (and), and both integer and floating-point addition (add, fadd) and multiplication (mul, fmul). We run each instruction in a loop on all available P-cores on each test device

What they did is define a handful of known workloads, with very different power profiles. And then they find that they can tell them apart by looking at the power of the chip. Well, duh.

I guess I'm describing some kind of custom in-memory database engine.

Anyway, I'm confused as to how the purity of Lisp functions would or would not prevent me from writing this code? Is this a use case that is bad for Lisp, or am missing something?

Another way to think about it is where does your car spend its time, 24 hours a day? Is there any time where you car is somewhere for an hour or more, sitting idle? That's where chargers should be.

The cost of computing these ads would be a lot more than today's keyword-based approach, that's certainly a problem. But think of hyper-relevant ads, based on the chat itself. There's a lot of information there, that beats tracking people's behavior online all day.

The article spends several pages to explain hexadecimal, bases, etc. Probably some fraction of the audience already knows that. In that case, the article should automatically adapt and skip that section. Some people will react negatively to the mathematical formulation, preferring the intuitive section that follows instead.

But this is a static blog post, so it doesn't know how to adapt. Imagine the same post, but with some toggles/sliders (you can come up with even more sophisticated mechanisms): I indicate my level of competency, and the article re-writes itself to match my knowledge. Skip the boring math, show a lot of examples, etc. Or the opposite. The point is that it adapts to you. GPT (or some future variant) is really good at doing this.

  une hache   - ma hache
  une horloge - mon horloge

Of course, Google is a giant today with a history of machine learning innovation. So they have a good chance of being successful in that new world. But the point is that many other companies get a chance again, which hadn't happened in 20 years. You couldn't displace Google in the old search/keyword/click business model. Now everyone gets a fresh start.

Who knows what the economics will be. Just like page rank early on, it was expensive to compute. But the money in advertising made it worth it, and Google scaled rapidly. Which language model do you run? The expensive one, or the light one (notice how Google in this announcement mentions they will only offer the significantly smaller model to the public). Can you make this profitable?

Other fun questions to answer if the industry moves to chat vs. search, in a 5-10 year horizon. What is the motivation to write a blog post by then? Imagine no one actually reads web sites. Instead a blog post to share an opinion, I'll probably want to make sure my opinion gets picked up by the language model. How do I do that? Computed knowledge may render many websites and blogs obsolete.

- there is a branch instruction every 10 instructions or so

- if you can't predict, assume you guess wrong half of the time

So every 20 instructions or so, you take the wrong branch. Sounds like it could be a significant overhead, from 10% - 50% depending on what the code inside the loop is doing.

For instance, if your code is doing only arithmetic operations, no load/store, such as computing factorial or other silly integer math, 20 instructions would probably take 20 cycles or less to execute, but the branch failure would cause a similar delay. So that's 50% overhead. In the other case, assume the 20 instructions are fairly slow (lots of memory accesses), then your branch penalty won't feel so bad.

My test went like this:

Q: Do not use the imperial system

Q: [some question which involves distances]

A: blah blah 1500 kilometers, which is 1000 miles blah blah

Q: I told you not to use the imperial system

It apologized, and repeated its previous answer, correctly omitting the miles data.

If you asked me to write a program that does that (without using ML), I'd have no idea where to start.

Wifi 6 bandwidth: 9.6 Gbit/s M1 SSD read speed: 6000 MB/s

So the read speed is about 6X faster than the Wifi 6 bandwidth.

You know, that kind of code:

  if(start) {
    if(!running) ...
  }
  
  if(stopped & !jumping) {
  } else if(running) ...

Frankly, with such low numbers, I would not draw any conclusion at all. Within the margin of error, the human could have detected 0, or maybe 2 cheating students. Who knows. It would change this results dramatically, so you can basically ignore them.

Also, the cherry on the cake is that the human also detected one cheating student who wasn't cheating. So human vs. software, no one wins.

I wouldn't. I have seen first-hand so many projects that took weeks of efforts, because we assume the person making the requirements used perfect language, when they could have been done in a day instead. One word can be the entire difference.

Maybe it happens to you in a big company: "the VP said '...'", so you must do exactly as the VP said. In practice, it turns out that the VP, just like Mary, could easily agree they didn't care if it's median or mean, they just need some number that summarizes the data.

Understanding the customer is key.