hajile

Water usage may be a drop in the bucket nationally, but they tend to put datacenters in areas without much water (often because evaporative cooling is less expensive in arid regions) and the effects to those local water supplies can be extreme.

The markets may not be claiming a recession, but the real world certainly is.

Vectors are pretty big for speeding up stuff like string comparisons.

Bit manipulation offers up to almost 10% advantage.

Zicond allows branchless code which represents significant speedups.

There’s also serious gains to be had from crypto support.

I’d guess the rest aren’t as important to Python, but those are quite important.

Rewriting critical stuff like hand-written SIMD takes a long time (though not as long as it took ARM because ARM had to overcome everything being written in just one ISA). Even a decade in, there are still lots of low-hanging fruit in RISC-V compilation that is still being dealt with. There's quite a few really great talks about the progress here and I think we're going to continue to see major gains in RISC-V performance via compiler changes for quite a while longer.

If such an instruction is so obviously important, we'll see it start popping up as a custom instruction in some cores then it will get adopted as an official extension by other companies trying to stay competitive.

> The problem is that RISC-V is never going to win at the high end. RISC-V will never have as much money driving it forward as x86 and ARM. As such, the blocks and integration will always lag.

Qualcomm recently bought Ventana and are looking at paying ARM billions (in addition to their current billions) for the privilege of designing their own cores. They sit on the RISC-V consortium and made proposals like Znew.

We could be seeing a high-performance RISC-V release from Qualcomm quite soon (especially if they can simply swap out the ARM decode for RISC-V).

Here's a good one.

Qualcomm beat ARM in court and reportedly pays 2-3% royalties where ARM was demanding 5-10% royalties.

Qualcomm's ARM license expires around 2028 with an option to extend to 2033 (for some amount of money). If Qualcomm is locked into ARM when renewal comes up, ARM is going to not only name the higher price, but likely charge even more to recover their lost revenue.

Qualcomm is already designing their own cores and ARM is charging them billions for the privilege. Increasing net profits 2-3% for simply doing the thing you are doing seems like a very easy choice. Increasing net profits 5-10% (maybe more) after the price hike seems like a fiduciary responsibility.

This isn't just talk either. Qualcomm already proposed a RISC-V Znew extension (nearly 300 pages of changes to make the ISA more like ARMv8). They bought Ventana (a company already done with their second very wide RISC-V design). They partnered with other companies to make Quintauris for promoting RISC-V too.

There's a non-zero chance we see a RISC-V design from Qualcomm before 2028 and I believe a near 100% certainty of a RISC-V design before 2033.

> I will make one ISA propsal: x86_64 should have had 32 instead of 16 registers

APX proposes exactly this along with 3-register syntax and some other things.

There are two big issues IMO.

1. It will take at least 15 years before most software ships with this because unlike something like AVX where you typically just rewrite a small part of your code that needs AVX, APX requires a 100% rewrite to take advantage.

2. APX instructions require an additional byte each time you use them compared to current instructions. I think there are still savings to be found, but they won't be as big as it might seem at first.

Lack of serious incentive combined with decades-long rollout seems a recipe for non-adoption at a time when RISC chips offer these features now.

Not too long ago, the claim was that having multiple Phds worth of knowledge was the real value of AI, but now that's gone and it's suddenly a good thing.

That's a sales tactic -- not a logical position.

I think this has to do with parallel decoding. How do you tell that the immediate is an immediate instead of an instruction? You have to carve out a very large part of the encoding space and you still can't fit a full immediate (eg, if you decided that all instructions starting with 1 were immediates, you'd be dedicating half of your encoding space to immediates and still be one bit short).

RISC-V does a 20-bit LUI (load upper immediate) then a 12-bit addi to the same register for the lower bits. Having access to 40-60 bit immediates makes 32-bit immediates a lot easier (with 64-bit immediates being multi-step, but quite uncommon).

Not traditional VLIW per-se as packets wouldn't imply parallelism (though that's theoretically possible) and instruction count would vary.

2-register to 3-register also just involves different wiring and costs nothing. I think you'd see 15-bit stick with 2-register. 20-bit would more interesting. You could choose to spend 3 bits on a third register or you could widen 2-register instructions to access the 32 core registers (or something between where you do 3-register, but only on 16 registers). 20-bit also reduces some of the need for very large 15-bit immediates (especially jump which is upward of 10% of the total space on 32-bit designs) which could allow more 15-bit instructions further improving effective density.

Easy access to 40/60-bit instructions mean stuff like vsetvli could simply go away and very useful instructions like FMA4 (instead of FMA3) could be added. Vector masking is another big one. They don't have enough bytes for a full vector mask set resulting in some hacks.

The big question is about jumping and predicting inside packets. You can add 2 bits for what externally looks like 16-bit addressing (where the 2 bits indicate packet position to jump to) or have faster jumps that always hit the beginning of the packet (at the expense of code density due to nops). There might even be a hybrid approach where short jumps can jump within a packed, but long jumps must jump to packet boundaries (which makes sense as most compilers make functions align on cache line boundaries anyway). There is a point for eliminating 20-bit (and all that compression goodness) for 45+15-bit pairs instead) as branches inside packets are immediately calculable.

How about EPIC-esque packet-based instructions?

64-bit instructions with 4 bits indicating instruction formats (60-bit, two 40+20-bit variants, 30+30-bit, 20+20+20-bit, three 30+15+15-bit variants, and 15+15+15+15-bit). Have each larger instruction type be a strict superset of the smaller instructions, but with larger immediates, more registers, and maybe additional instruction formats (eg, for SIMD).

Something like that would be even easier to decode (converting short instructions to long is simply a bit of wiring). Instruction density should increase due to 20-bit instruction type. Having properly-aligned instructions would help with fetching performance. Larger instructions means you can jump 4x further with the same immediate and 16-bit offsets. No need to have some of the V extension workarounds (from not wanting to add 48-bit instructions).

How can you use consistency and SSE/AVX in the same sentence?

SSE has inconsistencies like SSE4.x vs SSE4a. AVX is an even more mixed bag. There are some 19 AVX-512 extensions and ZERO chips support all of them.

The situation is so bad that AMD and Intel got together to make AVX10 to unify everything. That seemed great, but Intel now has AVX 10.1 and 10.2 in addition to the base set, so there we go again...

x86 is a massive battleground with tons of competing extensions like FMA3 vs FMA4 (why did FMA3 win???) and in cases where one of the competing variants didn't win, we get something like virtualization extensions being completely different between Intel and AMD. There's also the rash of security extensions that have gone through various support and dropped support (not to mention using some of this stuff for market segmentation and further fragmenting the ecosystem).

x86 is anything but consistent if you look into its history (or even it's present).

Neither AlphaGo nor AlphaZero were transformers. Why would you expect the same results?

Going further, current LLMs have at least an order of magnitude more computing resources, but completely suck at go. Why would this suddenly change unless we dumped the countless games played by alphago for them to train on?

Scheme SRFIs make Scheme a much bigger language while also making it painful to use because you have to research which SRFI are implemented in your scheme variant then map those SRFI numbers to what they actually do then you usually need read the raw SRFI because that's the only documentation around.

R7RS is going on 14 years old and there's still no R7RS-large either. They really need to just take the critical SFRIs, give them names and proper documentation, then call them R7RS-large and move the current work to a future R8RS.

JS converts most number into 31-bit ints internally (which is why it’s so fast). GUI work often prefers int or f32.

You have a point about games, but the CPU is seldom the limiting factor.

Very few people are handling raw audio. For the rest, codec hardware beats everything else by a massive margin.

Being good at floats matters, but even float-heavy code generally has way more integers than floats making integer performance king.

> Itanium would still have sucked

I'm not completely convinced of this.

If you ignore VLIW, you just have a very unexciting RISC ISA, but because of the VLIW, you get extra scheduling info that most RISC designs don't provide which might be advantageous. The real question is actually about the code density of 41-bit instructions and if it can be offset by the 128-bit package (and perhaps something like allowing new 24-bit compressed instructions).

Poulson already somewhat proved part of this as it added back a traditional frontend and even added some OoO capabilities and 4-way SMT. It wasn't earth-shattering, but it wasn't absolute garbage either.

How will costs come down?

Each generation of model is more expensive to train and run than the last.

More efficient hardware? That means throwing out billions of dollars worth of existing hardware and buying billions more in hardware. The cheapest hardware is the stuff they already have because spending $70B in hardware to halve a $2-4B electricity bill just doesn’t make financial sense (and that’s if doubling hardware efficiency can happen before they go under).

If they cut costs by using smaller models, they are then racing to the bottom vs China which seems hard to do (especially with China’s cheap solar energy).

As it stands, the leaked financials prove him correct. Given their losses (not counting restructuring) they are going to need enough funding to buy any of the bottom 300-400 of the F500 companies outright just to keep the lights on for the next year.

To say that’s an outrageous amount isn’t overstating in the slightest.

Given they are already being forced to drop inference prices in an attempt to compete with Chinese providers, I don’t know if billionaire investors will be willing to hand out that much money this time around if they can’t generate enough value to break even despite the hype.

The real answer is to stop trying to have AI reviewing AI code because it's just the blind leading the blind.

Nvidia already seems to be pivoting to "Local inferencing" with stuff like RTX Spark laptops.

The problem is that local inference machines won't allow anywhere close to their current margins or gross sales figures. At the same time, the huge overbuild of GPUs is going to crash server sales and prices for around a half-decade as companies try to avoid hardware upgrades or buy cheap, used equipment to save costs.

I think Nvidia will survive, but they'll be back to something like a 1T (or lower) valuation.

On a $100B AI datacenter, some 60-70% of the center needs to be replaced every 2-6 years. These companies claim the hardware lasts 6 years while simultaneously claiming they are relying on new hardware to lower inference costs (implying much more frequent updates).

Over 40 years, that's 7-20 hardware changes, that turns into between $420B and $1.4T of ongoing investment (not accounting for inflation). The $30B that is called "infrastructure" only accounts for 2-7% of the overall bill.

This is NOTHING like fiber buildouts because the fiber lasts the whole 40 years with ZERO replacements and very close to 100% of the cost is infrastructure rather than a tiny percentage.

All "interim" AI contributions to OpenJDK can and will be sued to make big money off of Oracle if those court cases don't go a certain way.

This is ironic given how Oracle has bet almost everything on AI succeeding.

The distillation commentary is really crazy coming from a company that stole all it's training material.

You formally verify that your incorrect solution executes without a hitch, but you might just be formally verifying that any user can hit your API and download all your plaintext passwords.

Lots of security bugs are caused by incorrect specs from misunderstanding the problem and a formal verifier can’t fix these. Humans can’t think through every situation either (or the bugs wouldn’t exist) SMS that’s doubly true because NOBODY understands all the interactions of the system as a whole and how changes in the part they understand affect everything else.

I’m not saying formal verification is bad, but it’s not a panacea and I’d wager would only fix a small percentage of existing bugs.

He sees what is coming. Crowd funding HAS to be dropping off as people choose between feeding their family or their favorite content creators (the creator will be fine as they have lots of other patrons is a guilt-free rationalization too).

I might buy this if those languages were actually faster. Instead, implementations of scheme with no commercial backers run circles around scripting languages with huge budgets.

Common Lisp can match and exceed languages like Java without the backing of a megacorp.

Carp Lisp is basically identical to C in performance and old PS games were shipped in a proprietary dialect (GOAL) and had very good performance.

Lisp has proven itself capable great performance at every abstraction level.

Last I checked, both of these were built on top of Common Lisp/SBCL.

function can do what a macro does, but it must run every single time while the macro can run just one time during the final code generation.

Let's say you have some Java-style a + b. It needs to work on native strings (concatenation) along with various kinds of ints and floats. The issue is that each one of these works differently internally, so the system is going to look at the incoming symbols and metasymbols (inferred type data) and is going to dynamically change between different functions like ADD_STR, ADD_INT8, ADD_INT32, ADD_FLOAT32, etc. If it did this at runtime by introspecting the type with some `switch(incomingType)` statement which involves all kinds of extra data and branches that clog the cache and create branches (two of the worst things you can do for performance). Instead, the macro (though they probably don't call it that) looks at everything and hardwires the correct output.

This hardwiring and its associated performance is the difference between a function and a macro. A macro `foo!(a b)` can inline into the current function while the function alternative must save all the registers to the stack and create a new stack frame which is very expensive.

The beauty of lisp macros is how functions and macros look the same. You may think that `(+ a b)` is the same as your favorite languages' `a + b`, but most lisps also allow `(+ a b c)` too. `+` is a macro rather than a function and it knows that `(+ a b c)` needs to convert to something more like `(+ a (+ b c))` and a typed lisp may go further to something like `(+f64 a (+f64 b c))`.

Your lisp criticism is orthogonal to the idea of lisp itself. Lisps like Coalton (lambda calculus) or Shen (sequent calculus) are strongly typed.

AI is an interesting point too where being closer to a raw AST is likely an advantage because it can focus more on the semantics instead of the syntax.