Hacker News Reader

Comment by rent0pat | Hacker News Reader

I've looked at the "only 18,935 lines of code" python code and it made me want to poke my eyes out. Not sure what's the point of this extreme code-golfing.

View on HN

From the instructions for potential contributors:

No code golf! While low line count is a guiding light of this project, anything that remotely looks like code golf will be closed. The true goal is reducing complexity and increasing readability, and deleting \ns does nothing to help with that.

https://github.com/tinygrad/tinygrad?tab=readme-ov-file#cont...

I think it's an amazing experiment.

You can look at the PyTorch code base and understand a small local function instantly, but if I would have a task to have a deep understanding of either PyTorch with all kernel code and low level code, whole CUDA code base + LLVM compilation code or Tinygrad I would pick Tinygrad in an instant.

The code looks hard because what it is doing is hard, but all its layers can be debugged.

Does doing hard work mean it shouldn't have any docstrings or comments? Cause that seems like an arbitrary way of making it "harder".

Docstrings and comments are amazing to be able to work with huge code bases with big teams where you don't want to understand the whole code base, because it's impossible.

If the goal is to not abstract anything away, and always look at the implementation itself to be able to change it / rewrite it, keep it tiny and it is able to beat PyTorch+CUDA+LLVM with 1000x smaller codebase, who am I to judge?

yes, it's really crazy, if people think you're exaggerating look at this:

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/co...

One of the cases why I think obsession with lines of code is one of the most counterproductive metrics, it always produces code like this.

That doesn’t look super awful to me? Hardly extreme code golfing.

The far more interesting part is the order of magnitude. If they can pull off a 20k LOC with zero dependencies (implying a pretty concise project size) and it still works well on meaningful applications, that’s pretty neat. A 1000x reduction in code size and matching/exceeding in perf is worth looking at. Probably also implying a better architecture as code golf isn’t gonna get you 1000x less code. Again - their claims not mine, so we’ll see.

But at that point they can triple the LOC to 60k with nothing but white space, new lines, and comments, for all I care. It won’t even add a zero.

it looks dense but perfectly readable. arguably more readable that way than if it had a bunch of extra new lines, definitions, and code blocks spreading the logic out into a larger visible area.

I'm inclined to agree. While not always appropriate or popular, it makes some sense to me to have the visual weight/area of the code being ~proportional to it's significance. Communicates the ideas more clearly. Instead of spending the majority of space telling me what _isn't_ going to happen, and hiding what is.

I often find myself wishing this was more ergonomic in languages.

It’s not crazy at all, but personally I like simple code that flows down the page more, not across

It's two-dimensional code, not one-dimensional code.

Declarations flow down the page, definitions flow across.

Running a formatter on that codebase would probably at least triple it.

Depends on formatter's config

Looks fine to me.

I'm going to guess that you're used to codebases that use an autoformatter.

There's definitely a case to be made for autoformatters in projects with a very large (>100) number of developers, but the price you pay for that is much poorer information density per screenful of code.

For things like tinygrad or sqlite with a small dev team it's possible to use whitespace, style, and formatting to communicate important information, just like mathematicians do. The AMS journals don't autoformat mathematicians' equations!

> I'm going to guess that you're used to codebases that use an autoformatter.

I'm used to Python code being "Pythonic" - which is one of those "I know it when I see it" terms.

Even with autoformatters, it's a choice of what criteria you pick. It's possible to have consistency without enforcing low information density.

I wish. Stuff like Prettier is purposely unconfigurable and leads to the unreadability of JS code across the ecosystem.

No, it really isn't. Like I said, math journals don't autoformat the mathematicians' equations. There's signal encoded in the formatting that a tool can't understand.

But if you have hundreds of developers on a project it does in fact make sense to sacrifice that signal in order to keep them from fighting with each other over silly formatting choices.

to quote an old academic research programmer: "Two-letter variable names are fine, as long as both letters are meaningful" :-)

Doesn’t seem too bad to me. It reminds me a bit of Clojure. Like Clojure, though, the lack of static types would be a headache.

You’re getting some pushback but one look at pm_reduce_collapse and I’m glad I don’t have to work in this codebase.

I see the advantage of the inline lambdas, but also it seems like it would hamper the ability to test the functions which can often have intricacies that catch you out. I believe one of the principles of Tinygrad is that tests shouldn't be part of the core codebase and shouldn't add to the total linecount.

I think there is an opportunity there to have a tag in comments for tests.

Have a hotkey to generate an id so you could have

  # lift x+y out of reduce on lt               :testID: color_adjective_noun  
  ((UPat.var("x")+UPat.var("y")).or_casted() < UPat.var("c"), lambda x,y,c: (x < (c.cast(y.dtype)-y)) if no_range(y) and no_range(c) else None),

and it creates a test that exists away from the main codebase. A process that runs tests could scan the code for testID's, intelligently extract the relevant part of the code into the test harness and runs the tests. There would need to be some infrastructure needed to handle placing the code to be tested into the right context but you could have

  1. A codebase that contained no active code required simply to facilitate tests, 

  2. All test tags auto removable for distribution builds

You don't really want or need classical unit tests for compilers. You need a harness that you can feed a program representation into and get a transformed one out, then check the result -- ideally mostly just by running it, or alternatively by comparing the output against a reference.

How is that not a unit test?

I've hear people refer to it as an end-to-end test, where unit tests usually test a single class or function.

That's only if you subscribe to the London school of though, which results in a bunch of useless tests that are coupled to implementation details.

I agree. It is definitely readable but clearly formatting decisions were made to keep line count down over the flow of code. It isn't even just the variable names, things like having a function with a single return statement on one line, while legal, I cannot imagine why someone would do it in a non one off script code.

> having a function with a single return statement on one line, while legal, I cannot imagine why someone would do it in a non one off script code.

I agree. Seems to be a personal preference, but still feels odd.

Huh. This is more readable than almost any large python codebase to me at first glance. It’s 2D, and well commented. I find it an excellent use of screen real estate. Perhaps it would be a different story if most of us weren’t stuck in 16:9?

I think the discomfort I felt at looking at that was because of the stylistic difference. There isn't any part of it that looks terribly hard to understand when compared to other code. I presume anyone dealing with this codebase would know what ctx entails and most things seem to logically flow from what it provides.

I think if I dealt with this on a daily basis I might end up finding if vastly more preferable, albeit at the risk of alienating fellow developers. In that respect it seems much like learning to use Emacs or vi

TBH that looks pretty decent to me. GPU programming is very very complicated. A simple kernel in torch is hundreds to thousands of LOCs, with much worse style.

I agree with some poster above, the code looks hard simply because the problem is very hard, not because they want to write it that way.

I don't think this is bad, if you know Python then most of the code will be fine for you. I think you're probably referring to pm_reduce_collapse, but while it looks daunting at first, it really isn't when you consider the alternatives, I'd be curious on how you'd improve it short of creating a DSL.

Is is code a form of AST rewrite rules for optimization? This operation still looks like a incomprehensible wall of code 40 years later (I looked in the past inside the C++ compiler).

This is like a JIT, generating code (machine code?). It’s a complex operation and IMO the code doing it is very clean and clear.

It helps that a single person can read all related bits on one screen.

Good code but on the way to terse Better comments needed

Funny to me that it is called simplify.py

Down with large line counts!

LLMs have done two things for us: made lines of code a commodity, but arguably even more important: made understanding an arbitrary amount of lines of code a commodity too.

with this in hand, large codebases that were designed in the primordial era of requiring human cognition to grok are emabarassingly antiquated.

"umm what does this mean?" -> right click -> explain with AI -> "got it".

I literally reddit-soyjacked when i saw "tackle LLVM removal". I would argue that LLVM was/is the most well thought out and well-architected piece of software that exists today... but in the age of AI... who could possibly care now? when I say "well thought out" and "well-architected", that is in the context of human eyeballs. But who needs human eyeballs when we have much more powerful computer eyeballs? who grok code way better and way faster than we do.

now, the massive line count of projects like LLVM, Linux, and Chrome become much harder to defend in the context of code re-use. Why drag in millions of code when you can get an LLM to write just the bits you care about? LLVM has a ton of optimisations that are all well and good, but how many of them actually move the needle on HPC platforms that do an awful lot of instruction re-ordering and re-scheduling anyway?

Could we get away with 5% of the codebase to do "just the bits we care about"? in the past the answer might be, "well yes but its still a lot of (manual) coding", but now - why not? Let an LLM loose on the design doc and fill in the gaps!

> HPC platforms

GPUs don't reorder instructions at all.

Reply on news.ycombinator.com