StefanKarpinski

Thanks for the bug report (assuming that’s you that filed it). The change that broke this was made first in Julia’s libuv fork but is being upstreamed into libuv main. It simulates UNIX chmod functionality on Windows, which is (apparently) tricky to get right across all versions and all corner cases. Supporting that correctly has been an outstanding TODO in libuv for a long time — what you’re seeing is Julia driving the development of libuv and therefore hitting the bugs first.

This issue was reported seven days ago and is now slated to be fixed in the next release of all affected versions.

Do you have a link to an open issue tracking this problem?

Several Julia core devs are also libuv maintainers or contributors. The main reason for the divergence is that Julia’s libuv fork has a significantly more flexible event loop model that allows using libuv from multiple threads efficiently. The main libuv project has been reluctant to accept that change since it’s a quite advanced capability that Node doesn’t need.

We considered juxtaposition with string literals for concatenation for a while, as in `file ".txt"`. That would make `a "" b` a natural string concatenation syntax, which rather like it, but Jeff felt it was a bit too clever/weird, so we didn't do it.

Technical enforcement would be nice, but it's a hard design problem and my point is that even with that, you still need the cultural aspect. Time and time again, we see that culture trumps technology for these kinds of things. In this case, the proof is in the pudding: in Julia there is no technical enforcement of consistency, but semantic meaning of generic functions is broadly respected and generic code works. It would be nice to add some kind of protocols that support enforcement, but it's (apparently) not necessary and it's a hard design problem.

> For example, the language does not let you overload + for a non-commutative operation

It does allow it? I'm not even sure how one would disallow non-commutative operations. How would you know if a definition is commutative?

This is an important point: abstractions like generic functions are only as good as how much people respect them, and enforcement of that is largely cultural. You can add technological levels of enforcement, but those are limited at best because people are brilliantly devious and can find ways to break any technological enforcement if they set their minds to it; the key to preventing that is convincing them to set their minds to respecting abstractions instead.

For a negative example of this, C++ introduced function overloading — which is a static and therefore broken version of multiple dispatch (that calls the wrong “methods” based only on static type information). They then immediately decided to abuse the bitshift operators for I/O — in the standard library, no less. (There are other abuses of overloading but this is the most famous one.) So that didn’t exactly create a culture where people respect the semantic meaning of overloaded functions. As a result, C++ has given function overloading a really horrible reputation. Partly because it is likely to be not actually so what you want because it’s static rather than based on the actual types of arguments, but even more so because there’s no culture of semantic consistency in C++, starting from the standard library itself — you cannot trust anyone to respect the meaning of anything, not even the language authors.

In Julia, on the other hand, we’ve always been very strict about this: don’t add a method to a function unless it means the same thing. We would never dream of using bitshift operators to do I/O. Since meaning is the level where human thinking operates, this makes it reasonable to write generic code that works the way you meant it to: you can call `+` on two things and just trust that whoever has implemented `+` for those objects didn’t decide to make it append a value to an array or something wonky like that. Not that people haven’t proposed that kind of thing, but because of the culture, it gets shot down in the standard library and elsewhere in the ecosystem.

But yeah, it’s hard to see how this would happen because there is nothing technical preventing the same kinds of abuses that are rampant in C++, it’s just the invisible but extremely force of culture.

Thanks, man :D

Many practicing statisticians do actually care about easily using GPUs and doing distributed computations on distributed data sets with the same code they use for a local data set, which is what those Julia capabilities give you.

I.e. MIT license

I don’t really get the first issue. If an algorithm doesn’t need to mutate a matrix then it will work for a mutable or immutable matrix. If it needs to mutate a matrix then how could it work for an immutable matrix? If you want a polyalgorithm that uses a mutating algorithm on mutable matrices and a non-mutating algorithm on immutable matrices, then you can just do that (and multiple dispatch makes it easy). What additional features are needed? Is there some language that “does this right”?

Is this in any way distinguishable from just picking random accounts? There is no discernible similarity between the supposedly similar accounts that I can discern.

Thanks for explaining that!

A later message claims (outraged at being accused of submitting intentionally broken code to the kernel, despite having previously done exactly that) that the patch was generated by a static analysis tool. Ok, what tool? How did you run it? The message where he claims this has since been deleted (by who? Edit: probably never sent to the list, see below) but here is a message from Greg KH which quotes it:

https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah...

You can see that Greg KH is skeptical that the patch was generated by a tool. It’s also unclear to me if the patch is actually harmless or not. It would be good to get some definitive clarity on that but the lkml discussion of it seems inconclusive.

I would add that the student’s tone in this message feels really familiar as an open source maintainer: this is the tone of someone doing something they know is bad being called out on it and trying to deflect with outrage. The last time I got that tone, it was someone who had created multiple sock puppet accounts to try to discredit my project and force moderators to reverse or apologize for a (completely justified and mild) moderation action. So I can’t help but feel that something fishy is going on here and the UMN research group still isn’t being honest about it.

The thing I’m still missing is a detailed explanation of what the heck was going on with the recent bogus commit that triggered the banning. Supposedly the “hypocrite commit” research was all done in 2020 and is now in the past. So what was going on with this latest bad commit? The student who submitted it claimed it was generated by a static analysis tool, which kernel maintainers have plausibly called bullshit on. Was that student lying? If so, what were they actually doing? If not, it seems absolutely necessary at this point to prove that they were telling the truth by publishing how that commit was produced, including the tool’s source code and how it was invoked. Even if the campaign to intentionally introduce security flaws was over in 2020, the more recent issue is that the same research group submitted an apparently intentionally incorrect patch and then seems to have lied about its provenance and why it was submitted. Until that’s cleared up any kind of apology feels premature and impossible to evaluate. How can anyone decide if an apology is sincere without understanding what was done?

This is great, I hadn't seen this before. Semantic Lawful / Syntactic Chaotic seems like a fair characterization of Julia. There's too much syntax for hardcore lispers to be entirely comfortable but almost all of the syntax is just method calls in disguise, so it's semantically pretty clean.

The trouble with that is that inherited wealth replicates existing inequalities, whereas newly earned wealth at least offers the chance for social mobility. I'd be more interested in median wealth of each generation.

Many years of being on the receiving end of issues shows that there are a large number of different potential problems which manifest themselves in similar ways. Some are fixed, some persist, some may be new. If people don’t file issues describing exactly what they tried and what happened, this vague complaint just goes into the bucket of “Who knows? Hopefully someone else files a proper bug report.”

I’m not sure what issue you think you identified by googling such a short problem description, but it seems like it could be any of:

- slow internet connection

- firewall / proxy issues

- antivirus gumming things up

- file system being slow when dealing with lots of small files (Windows mostly)

- precompiling Plots took longer than expected

- precompiling Plots hit a deadlock

- loading Plots took longer than expected

- loading Plots hit a deadlock

- something else?

Worse, what “stuck” means is also ambiguous. Does that mean it failed with an error? Does that mean a download started but then was too slow for the user’s taste? Does it mean that a download started but never got any data at all? How long did the user wait?

My best guess is that git cloning the registry on Windows is taking a long time and isn’t actually stuck. There’s a fix for that being worked on for 1.7 (don’t unpack registries).

Not my best but as I said, I was genuinely confused about the benchmarks.

Very doable, “just” a bunch of moderately tricky compiler work. Will happen at some point. Things that would make it happen sooner: someone interested in compiler work decides to do it; some company decides to fund it.

Hi, Oscar! Nice user name :P

Has a version of Java with value types been released?

When I wrote that I was under the impression that the brainfuck interpreter implementations were the only benchmarks in the repo. There are, however (I now realize), also benchmarks for base64 decoding, JSON parsing, and writing your own matmul (rather than calling a BLAS matmul, which is not generally recommended), so this is more reasonable than I thought but still a somewhat odd collection of tasks to benchmark. Of course, microbenchmarks are hard — they are all fairly arbitrary and random.

In a delightful twist, it seems that there is a Julia implementation of a Brainfuck JIT that is much faster than the fastest interpreter that is benchmarked here, so even by this somewhat esoteric benchmark, Julia ends up being absurdly fast.

https://news.ycombinator.com/item?id=26585042

I love the Julia community

Please file an issue describing the situation: https://github.com/JuliaLang/julia/issues/new

This is very legit issue that the compiler team has their eye on and plans to work on.

I don't know that the object model is why writing a BLAS in Java doesn't make sense. After all they special case `float` and `double` as primitives, which bifurcates the whole type system and is its own whole issue, but means that you can store them efficiently inline. I'm actually not sure what stops someone from writing a BLAS in Java except that it would be hard and there's no point.

I'm not — memory layout and memory management are (fairly obviously, I would think) intimately related. In particular, pointer-heavy memory layouts put way more stress on the garbage collector. Java's choice of making objects mutable, subtypeable and have reference semantics, basically forces them to be individually heap-allocated and accessed via pointers. On the other hand, if you design your language so that you can avoid heap allocating lots of individual objects, then you can get away with a much simpler garbage collector. Java only needs spectacular GC technology because the language is designed in such a way that it generates a spectacular amount of garbage.

I love GC — it solves a ton of nasty problems in a programming language design with a single feature that users mostly don't have to worry about. Just because you have a GC, however, doesn't mean that it's a good idea to generate as much garbage as you can — garbage collection isn't free. That's where Java IMO went wrong. Java's design — objects are and subtypeable (by default) and mutable with reference semantics — generates an absolute epic amount of garbage. It seems like the hope was that improvements in GC technology would make this a non-issue in the future, but we're in the future and it hasn't turned out that way: even with vast amounts of money that have been spent on JVM GCs, garbage is still often an issue in Java. And this has given GC in general a bad name IMO quite unfairly. It just happens that Java simultaneously popularized GC and gave it a bad name by having a design that made it virtually impossible for the GC to keep up with the amount of garbage that was generated.

It is entirely possible to design a garbage collected language that doesn't generate so goddamned much garbage — and this works much, much better because a relatively simple GC can easily keep up. Julia and Go are good examples of this. Julia uses immutable types extensively and by default, while Go uses value semantics, which has a similar effect on garbage (but has other issues). With a language design that doesn't spew so much garbage, if you only care about throughput, a relatively simple generational mark-and-sweep collector is totally fine. This is what Julia has. If you also want to minimize GC pause latency, then you need to get fancier like Go (I think they have a concurrent collector that can be paused when it's time slice is up and resumed later).

Persistent data structures are a whole different question that I haven't really spent much time thinking about. Clojure seems to be the state of the art there but I have no idea if that's because of the JVM or despite it.

Obviously a 10-million-element array doesn't get stack allocated. But if individual objects of some type are immutable, then they can be stack allocated, or maybe not allocated at all (kept in registers).

Edit: reading your other post, it seems like you may mean persistent data structures, a la Clojure, rather than immutable structures, which are quite different. The former would indeed always be heap-allocated (it's necessary since they are quite pointer-heavy). Immutable structures, on the other hand are detached from any particular location in memory.

Moreover, if the elements in an array are mutable, eg Java objects, then each one needs to be individually heap allocated with a vtable pointer and the array has to be an array of pointers to those individually allocated objects. For pointer-sized objects (say an object that has a single pointer-sized field), that takes 3x memory to store x objects, so that's already brutal, but worse is that since the objects are all individually allocated, the GC needs to look at every single one, and freeing the space is a fragmentation nightmare. If the objects are immutable (and the type is final; btw all concrete types are final in Julia), then you can store them inline with no overhead and GC can deal with them a single big block.

Btw, I had to vouch for you to undead your posts in order to reply. Looks like you got downvoted a bunch.

I mean I’m not trying to hate on Java — pointer-heavy programming was all the rage when it was designed, and GC was a hot research topic, so there was good reason to be optimistic about that approach. But it turns out that it’s very hard to make up for generating tons of garbage and pointer-heavy programming hasn’t aged well given the way hardware has evolved (pointers are large and indirection is expensive).

It doesn’t, it just doesn’t have a $100B GC like Java does. Rather than spending that kind of money trying to compensate for a language design that generates massive amounts of garbage (ie Java), Julia takes the approach of making it easier to avoid generating garbage in the first place, eg by using immutable structures that can be stack allocated and having nice APIs for modifying pre-allocated data structures in place.