KenoFischer

Yes, it's useful when you're ssh'd into some production system somewhere and just want to get a quick sense of some data in memory instead of going through the steps of installing a full graphics stack (which fortunately is pretty much a one-liner in Julia also, but can take a few mins).

This is absolutely a step in the right direction. GitHub permissions are ridiculously coarse-grained. Hopefully this will lead to more fine-grained permissions on GitHub in general.

Dynamic mutliple dispatch takes into account the dynamic type rather than the static type as you would in say-C++ style function overloading. The single dispatch equivalent is function overloading vs virtual calls.

Just FYI, your 1038.pdf hyperlink at the bottom goes to the 975.pdf paper ;).

I'm not sure I really "got distracted" by building those tools. Building the tools was kind of the point for me and the class was simply a good way to focus the tools on a concrete effort. I acknowledge that it ended up occasionally getting me in trouble when forward progress on class projects was stalled on seemingly unrelated tooling and I got an appropriate admonition about it at the end of the year from the professor, but on the other hand, doing the class this way was way more impactful long-term. A lot of that tooling is still in active use across the Julia ecosystem, while the actual code it was used for at the time never got touched again after the end of the semester.

That's pretty much my general philosophy when it comes to writing software though. Whenever possible, figure out what tooling would have made the task easy and then build that and solve the problem simultaneously. You're probably gonna have to throw away the latter code, but at least the next time it'll be easier, because you now have the tooling. Even better if you happen to be in a position like mine where you already build tools for others, because that way other people will end up benefiting too.

Anyway, does it still occasionally get me in trouble? Yes. I not infrequently end up in situations where the software is essentially done, but I have to write a compiler pass to make it actually work, because I decided to write the code in a way that I thought it should be written, rather than a way that would be performant out of the box. But hey, at least I'm aware of it (https://twitter.com/KenoFischer/status/1529814861693259777) ;).

vchuravy's link gives the details, but basically, there's a microarchitecture optimization in Zen that breaks determinism of the performance counters. Fortunately, there's a chicken bit that turns it off, which is what the script does. I've been trying to convince AMD to officially document the bit such that the kernel can set it automatically, but no luck so far.

There is still one remaining annoyance, which is that AMD's NMI latency is super high, which directly tanks rr's reverse execution latency. There's probably some improvements that could be made to the replayer to be more aggressive about optimistic assumptions on NMI latency and retrying if those assumptions are off, but it'd be a fair bit of work. I don't really understand why AMD decided to use this kind of architecture. It also makes profiles much less accurate.

> Thanks for the honest assessment. What about correctness/ composability of compiler transforms like AD, reliability of GPU acceleration and predictability of optimizations? (basically what you've discussed in some of your compiler talks).

I don't think we really have a good answer yet, but it's actively being worked on. That said, I don't think we can be faulted for that one, because I don't think anybody really has a good answer to this particular design problem. There's a lot of new ground being broken, so some experimentation will be required.

> TPUs, kernel fusion, branching (Lux relies on generated functions) and copy elision (though this last part is being worked on IIUC).

We have demonstrated that we can target TPUs. Kernel fusion is a bit of an interesting case, because julia doesn't really use "kernels" in the same way that the big C++ packages do. If you broadcast something, we'll just compile the "fused" kernel on the GPU, no magic required. There is still something remaining, which is that when you're working on the array level, you want to be able to do array-level optimization, which we currently don't really do (though again, the TPU work showed that we could), but is broadly being planned.

> Edit: re : bandwidth issue Jan Vitek's group is thinking a lot about the verification vs flexibility tradeoff and some people are working on a trait/ static typing system. Maybe something can be done to help them along?

We work closely with them of course, so I think there'll be some discussions there, but it's a very tough design problem.

Thanks. Appreciate your thoughtful moderation as always :).

So this one is a tough one for me, because Yuri has certainly spent significant time with Julia and I think he's a very competent programmer, so his criticism is certainly to be taken seriously and I'm sad to hear he ended up with a sour opinion.

There's a lot of different issues mentioned in the post, so I'm not really sure what angle to best go at it from, but let me give it a shot anyway. I think there's a couple of different threads of complaints here. There's certainly one category of issues that are "just bugs" (I'm thinking of things like the HTTP, JSON, etc. issues mentioned). I guess the claim is that this happens more in Julia than in other systems. I don't really know how to judge this. Not that I think that the julia ecosystem has few bugs, just that in my experience, I basically see 2-3 critical issues whenever I try a new piece of software independent of what language it's written in.

I think the other thread is "It's hard to know what's expected to work". I think that's a fair criticism and I agree with Yuri that there's some fundamental design decisions that are contributing here. Basically, Julia tries very hard to make composability work, even if the authors of the packages that you're composing don't know anything about each other. That's a critical feature that makes Julia as powerful as it is, but of course you can easily end up with situations where one or the other package is making implicit assumptions that are not documented (because the author didn't think the assumptions were important in the context of their own package) and you end up with correctness issues. This one is a bit of a tricky design problem. Certainly adding more language support for interfaces and verification thereof could be helpful, but not all implicit assumptions are easily capturable in interfaces. Perhaps there needs to be more explicit documentation around what combinations of packages are "supported". Usually the best way to tell right now is to see what downstream tests are done on CI and if there are any integration tests for the two packages. If there are, they're probably supposed to work together.

To be honest, I'm a bit pained by the list of issues in the blog post. I think the bugs linked here will get fixed relatively quickly by the broader community (posts like this tend to have that effect), but as I said I do agree with Yuri that we should be thinking about some more fundamental improvements to the language to help out. Unfortunately, I can't really say that that is high priority at the moment. The way that most Julia development has worked for the two-ish years is that there are a number of "flagship" applications that are really pushing the boundary of what Julia can do, but at the same time also need a disproportionate amount of attention. I think it's overall a good development, because these applications are justifying many people's full time attention on improving Julia, but at the same time, the issues that these applications face (e.g. - "LLVM is too slow", better observability tooling, GC latency issues) are quite different from the issues that your average open source julia developer encounters. Pre 1.0 (i.e. in 2018) there was a good 1-2 year period where all we did was think through and overhaul the generic interfaces in the language. I think we could use another one of those efforts now, but at least that this precise moment, I don't think we have the bandwidth for it. Hopefully in the future, once things settle down a bit, we'll be able to do that, which would presumably be what becomes Julia 2.0.

Lastly, some nitpicking on the HN editorialization of the title. Only of the issues linked (https://github.com/JuliaLang/julia/issues/41096) is actually a bug in the language - the rest are various ecosystem issues. Now, I don't want to disclaim responsibility there, because a lot of those packages are also co-maintained by core julia developers and we certainly feel responsibility to make those work well, but if you're gonna call my baby ugly, at least point at the right baby ;)

I have a patch that teaches GDB how to do this here: https://github.com/JuliaComputing/gdb-solib-wine. Unfortunately, I don't really have the time to upstream this, but perhaps it's useful or maybe somebody sees this and wants to take on the upstreaming.

I described what needs to happen in https://github.com/JuliaLang/julia/issues/40399, but so far nobody has had the time to implement it.

For those who like to reminisce, here is the HN discussion from 10 years ago: https://news.ycombinator.com/item?id=3606380

LLVM has this as well now, though I believe this blog post predates the LLVM implementation and may have socialized the concept in LLVM circles.

Julia Computing | REMOTE | Full Time | juliacomputing.com

Julia Computing is currently hiring for many roles, including

    * Cloud Engineer

    * EDA Core System & Compiler Engineer
 
    * Julia Application Developer
    
    * Scientific GUI and Visualization Engineer

    * Security Engineer

See https://jobs.juliacomputing.com/ for a full list. Note that this includes positions to work on https://cedar-eda.com, in case people are looking to help us fix EDA tooling :)

Most also available in internship format. That said, if you don't find any of these to fit your background, but you still have a strong systems or compiler background - please do shoot me an email - would love to see if we can make something work.

I'm assuming OP is referring to this: https://oncobites.blog/2020/07/29/modeling-aging-and-cancer-...

Mutation is tricky, because basically the only way to do it sanely is to copy all data into the residual, but since most arrays aren't actually mutated, that's extremely wasteful. I've been hoping to address this by changing mutability in the language more generally (e.g. https://github.com/JuliaLang/julia/pull/42465) to make immutable arrays the default at which point there wouldn't be a penalty anymore. I've had request to do the mutable copying optionally, but it's a bit tricky, because it needs rule system integration and the rule system currently doesn't reason about mutation.

As for exceptions and recursion, shouldn't be a problem, just needs to be implemented.

> basically the big takeaway that i walked away with was that they defined a sort of pseudoclosure in the guts of the language before the compiler which relaxes some of the behaviors of a closure (ie what gets captured) which then enables the creation of larger optimizable regions for the compiler in the context of doing autodiff.

Yes! That's essentially correct.

Maaayybee ... Writing a blog post always seems like a time commitment, and I really should be doing other things ;). Somehow sequeezing in a big HN comment while I'm waiting for my builds to go through doesn't feel like the same, but I'm probably just deluding myself ;).

Diffractor solves a couple of issues that are inter-related. I'm gonna contrast with Zygote which is the current "standard" AD in Julia and the most fancy one we have (though there are a couple other AD packages that are useful in certain situations that Zygote is bad at - Diffractor will cover some of these but not all).

Essentially, what Zygote does is insert itself into the compiler pipeline at the lowered code stage to perform the AD transform. That is it operates on the form of Julia code before we perform any type analysis, devirtualization or optimizations. Essentially, you can think of Zygote as a lisp-style macro applied automatically/dynamically to every function being called starting at a particular entry point. This works pretty well, but has a couple limitations.

The first is that it has absolutely no semantic information available (since it operates on non-inferred code), so it can't use that for optimizations or things like data layout planning, which are important optimizations for a production-grade AD system. Essentially, it's not allowed to know that the `+` symbol it sees is actually the `Base.+` function that does addition, so must make the most pessimistic assumptions. This issue doesn't actually show up that much in machine learning use cases, but it's a big issue when you need to AD scalar code (which happens frequently in various differentiable programming contexts).

The next issue with running at this stage is that a lot of existing julia code is written with some mental model of the capabilities of type inference and the optimizer. For example, destructuring code like `a, b = f(x)`, people don't think about at all, but semantically that allocates several tuples and then indexes into them to take them apart. By running the AD transform, you basically double (at least), the complexity of every operation, so in a number of cases patterns that used to completely optimize away are now terribly slow, because they are no longer optimized (and then AD transformed on top of that).

A related issue is that because you cannot interleave optimization with the AD transform, if you want to perform nested (i.e. higher order) differentiation, you're gonna get exponential code generation, which you then have to hope the optimizer will cut down again for you, which it often can't all the way, but even if it could, generating exponential code in the first place is bad, because you're gonna wait an exponential amount of time for the compiler to be done with it (and super-exponential in practice, because the compiler is not linear in the size of the input problem), so Zygote is basically unusable for anything beyond second order (and even at second order is a struggle).

Lastly, there is also a technical issue, which is that code that operates on lowered IR isn't technically allowed to build any closures, but the AD transform has to do that in order to put the code for the backwards pass somewhere. Zygote gets around this by taking advantage of the fact that the AD transform is pure, so it basically runs everything twice (once to generate the forward pass, once to generate the backwards pass when execution gets to it) and it knows that things match up because the input code is the same. That mostly works, but this dependency isn't visible to the runtime system, so you can get into issues where code is updated in between the forwards and the backwards pass, which breaks Zygote in all sorts of ways.

Anyway, Diffractor is designed to fix all of this by "simply" moving the AD transform stage from lowered IR to post-inference (where the optimizer sits). The issue is that Julia the language, currently doesn't really allow semantic transformations post inference (after all optimizers are supposed to make things faster, but not change the outcome of things) and in particular, running optimizers is always optional and the language may choose not to do it. So to get there, we need to do a couple of things:

1. We need to have some sort of wedge into the semantics of the language that allows for optimization-time changes that are semantic. For this, I added `OpaqueClosures`, which are essentially like regular closures, except that they do not have semantics as to their capture lists or the code that they run. Now, this may be confusing to people from some functional languages where all closures have such semantics (you can see SPJ ask this question in the linked discussion we had - didn't think about it ahead of time, because I'm so used to our semantics, so my answer was a bit muddled), but essentially in Julia, the contents of closures is semantically visible and optimizations over closure boundaries are mostly prohibited, because ordinary closures do not close over the world age (i.e. if you have `f = x->sin(x)` and then say `sin(x) = 9999`, then the next execution of `f` will return 9999). So opaque closure basically change this and say "nothing in the system is allowed to look at the code they contain, or the capture list, and also we capture the world age". More importantly though, we now have a datastructure (the capture list of an opaque closure) that is allowed to be changed by the optimizer, so we drive basically drive a truck through that.

2. We need the actual mechanism to move the transform to inference time. This isn't super hard, but we haven't quite finished this yet, and it's something I'm hoping to get to over the next couple of months. It's somewhat intertwined with making the compiler in general more accessible to packages outside the core language. There's about 8-10 different packages that want to do compiler-y things on Julia IR and we really need to figure out a good solution to make that generally possible. But as always, designing core language APIs is a bit tricky, because you're gonna be stuck with them for a while.

There's also some other nice bits and pieces in Diffractor. The big one is that I did a fair bit of theoretical exploration at the top of this project to really understand higher order AD. When I first took differential geometry (10 years ago now), I used to joke with my classmates that I had no idea what a second derivative was, because all the textbooks basically just say "look, the tangent bundle of a smooth manifold is a smooth manifold" and never actually really go into higher order derivatives at all. Anyway, I really sat down to work all that out and in the course of that I came across a way to do higher order derivatives more efficiently (under suitable assumptions of what the compiler does) than just nesting the first order transform. In retrospect, it seems pretty obvious, but I can honestly say that I didn't think about that until I worked out the theory. In the course of it, I also managed to make very precise the notion that reverse-mode AD and pullbacks of cotangent vectors are the same thing. This connection was pretty well known in the oral tradition, but I never really saw a convincing writeup of it (and my result extends to higher orders of course).

Other than that, it directly uses ChainRules.jl as its AD rule system, where Zygote used to keep its own rules (Zygote and ChainRules developed concurrently, and Zygote was later adapted to look in both places, but its a bit of a mess) and it also has a unified forward mode that (in theory, once it's robust enough), subsumes both ForwardDiff.jl and TaylorSeries.jl.

In theory, this should all make for a pretty good AD system, but there's a fair bit of work still to be done to make it robust (though all the pieces except the julia-level mechanism for actually moving the AD pass are done and tested) and as always, time is very limited :).

We had a debugger like that a few years ago, but the experience was unsatisfying for people, because you got the "debugging optimized C++ code" experience with unreliable breakpoints and mostly unavailable debug variables. I took the decision to scrap that and instead put out something simple that's slow but robust and reliable. The plan was always to then use the JIT on top of that to create a "debug-specialized" (using statepoints for local variables rather than DWARF) version of the running code, which should give you perfect debuggability at minimal runtime cost, but it's a fair amount of work that nobody has wanted to do yet.

In general, traditional debugging has always taken a bit of a backseat in Julia, because people code is usually decently functional, so they just run it in a Revise loop and write their state dumps directly into the code (you could deride that that printf debugging, but I think it has a bit of a bad rap, particularly in a live-reloading system, where you basically get an execution log of your revising expression on every file update).

There are still cases where a traditional debugger is useful, so I'm hoping someone will take that on at some point, but so far there've been higher priorities eleswhere.

Also do note that you can switch the debugger into compiled mode, which will be faster, but ignore breakpoints.

> Like, if i write a simple pure function in Julia and call code_llvm on it… How “standalone” is the llvm code (if that is even a thing)? When does GC get called? How exactly does the generated code depend on the runtime?

It's standalone unless there's explicit calls to the runtime in it. The most common runtime support is probably heap allocation, so if you see a `jl_alloc_obj` in there, that gets lowered to runtime calls eventually. GC gets called during allocation if the runtime thinks there's been enough garbage generated to have a collection be worth it.

That (but with Julia rather than XLA obviously) coupled to high performance data paths between the GPU and SDR.

Well, we're building - they're paying :). I tweeted some photos here for people who want to see our initial testbed (yes, this is completely off the shelf hardware - part of the point of the project is to see how far we can take that): https://twitter.com/KenoFischer/status/1412517017354178561. Happy to answer questions if people have them :).

Edit: The original title of this was "Julia Computing and DARPA to build ...", which is what this comment is referring to ;). Thanks for updating the title.

The agreement is available online, in case you speak French: https://legal-service.web.cern.ch/system/files/downloads/Acc...

It's probably not obvious to people outside the community, but the amount of effort that's currently going into MTK and the larger SciML stack is absolutely enormous. Depending on how you count, maybe an order of magnitude more than is currently going into Julia proper. By my count at least 30 talks at JuliaCon this year were about SciML applications. That whole ecosystem is really taking off.

This project is just starting, so I have hardware sitting on my desk and have used it a bit, but other than that not much.

Finance was a focus area early on and we have a fair number of consulting clients there and JuliaHub is available of course, but we were never able to figure out a dedicated domain-specific, non-niche product to sell into the space. Maybe in the future.

Sure, but there's two separate concerns here. One is that money will turn us evil and we'll exercise undue influence. My counterpoint was that JC has been around for six years now and in that time we've actually strengthened Julia significantly as an independent project. My other point though was about concentration risk. I think far more common than people turning evil is that companies go all out raising money, become the only people developing a project and then if the revenue doesn't come as planned, the company and the project fail together with unpleasant results. At this point, if Julia Computing fails, Julia the language will survive no problem. Of course we're not planning on failing, but before going out on this commercial path, it was hugely important for all of us that Julia is on solid footing. For must of us, what we've built in Julia is our "life's work" (ok, it's only been 10 years, but that's a substantial effort still) and we're not planning to let that just die.

These kind of concerns are not unreasonable in general of course, but in this case let me point out that Julia Computing has been a commercial enterprise for more than six years. Also, our commerical product is deliberately not Julia, but rather we're building our products on top of Julia, just like anyone else might. In fact there are several startups unrelated to us that have built multimillion dollar businesses entirely in Julia . At this point there's a bunch of companies that depend on Julia and are committed to it's future - we're just one among them.

Nothing complicated.

Stream 1: Build amazing products for particular domains, charge license fees

Stream 2: Build a great SaaS platform for running Julia, charge for compute

Since all of our domain products are built in Julia and often involve significant compute cost for their intended application, hopefully both at the same time :).