Hacker News Reader

Comment by wiz21c | Hacker News Reader

>> A new tool has been added to provide insight into the way that loading dependencies contribute to the load time of a package.

That doesn't sound like an improvement to the dependencies loading... Which for me happens every time I change a struct and must reload Julia (which happens a lot when I'm prototyping code, which is what I do 90% of the time)...

View on HN

> That doesn't sound like an improvement to the dependencies loading...

This gives the package authors a tool to basically "profile" the loading time of their package, which will help them optimize the loading time. So there _will_ be downstream improvement to package loading for us users too.

> Which for me happens every time I change a struct and must reload Julia (which happens a lot when I'm prototyping code, which is what I do 90% of the time)

The workaround involving `MyClass = MyClass1` that a sibling comment mentions comes from [this Revise.jl manual page](https://timholy.github.io/Revise.jl/stable/limitations/). Revise in general enables a few different ways to work around this, and it's useful to integrate into your workflow (as much as it feels weird at first to go from full REPL to half REPL-half editor).

I saw this beautiful hack to get around this:

```

struct MyClass1

...

end

MyClass = MyClass1

```

Then, whenever you need to change MyClass, just add 1 to the version number of MyClass in those two or three instances, and the repl will just compile MyClass2 as a new class to be used for all future calls in the repl.

Not good if you want to also know stuff about the algorithms performance as that assignment is now accessing a global vs accessing a const.

That is not dependency loading.

About your issue, things you can do: - Put the struct inside another module. - Use this package: https://github.com/BeastyBlacksmith/ProtoStructs.jl . It gives you a simple @proto macro that you can prepend to the struct definition and makes all changes immediate.

It seems to address my problem of constantly restarting REPL (and yes, it triggers dependencies reloads; although not an issue with dependencies directly).

If it was stackoverflow, I would have accepted this answer with a kind comment :-) Thanks !

> > This gives the package authors a tool to basically "profile" the loading time of their package, which will help them optimize the loading time. So there _will_ be downstream improvement to package loading for us users too.

It lead to https://github.com/SciML/RecursiveArrayTools.jl/pull/217 . 6228.5 ms to 292.7 ms isn't too shabby.

You can use a macro for your structs to make them changeable while prototyping. I think it is called @prototype or soemthing. You need a package for it. Google it.

There is RedefStructs: https://juliahub.com/ui/Packages/RedefStructs/nrk8Q/0.1.2

and ProtoStructs: https://github.com/BeastyBlacksmith/ProtoStructs.jl

I’ve never tried them, but they’re advertised as doing the job.

+1 for these packages; I’ve used both at different points.

Addendum for those who are new to this, the idea is to use these macros during development, and then get rid of them once the types stabilize (since they tend to have a performance cost).

Had it ever occurred to you - if this was a real issue how would anyone ever get any work done? The answer is modules.

module X struct etc... end end #module

Change without reloading.

But doesn't defining a new module lead to code recompilation because all the functions in the module are new? Or does Revise.jl have some way around this?

yes, but it will only recompile the code in the module. All the dependents will already be compiled.

Right, but there is a separate new feature which caches more lowered code.

https://julialang.org/blog/2022/08/julia-1.8-highlights/#imp...

The core devs also said that they are aiming to cache native code in the next year or so.

It is useful in determining which packages are dominating a given package’s load times, allowing for a better understanding of where to invest effort to reduce runtimes. This has already helped in directing efforts towards a number of common dependencies that increase load times…

This is very annoying but they recommend working in the REPL

Maybe I'm wrong, but I do work in the REPL. Each time I update a struct, I need to restart the REPL, hence reload dependencies... Of course, I could make my own sysimage, but it's a bit painful...

Is Julia "doomed" on this issue or is it just that they prefer to work on other use cases first ?

Julia's 1.8's load times are dramatically better than Julia 1.0's. It keeps getting a bit better with every release.

But there is a project to cache native code (without needing to build a sysimage) that may make a huge difference...

It's not doomed, no. It's possible to have a low-latency Julia. It's just been happening slower that one could have hoped, especially for the users who started using Julia and could tolerate the latency because they expected it would be better soon.

* It's been reduced by like 75% (roughly, not sure about the real number) since Julia 1.0, so material progress has been made.

* In the future (I'm guessing 1-2 years, of course uncertain), native code caching could make a huge difference.

* On a similar timescale, Julia could allow compilation into static binaries. This could possibly be done for some dependencies that does not need to be generic, which would take another chunk of the compilation need off.

Yes and no. There are ways around that for development like https://github.com/BeastyBlacksmith/ProtoStructs.jl. But changing the definition of actual structs isn't very feasable because what happens with the structs that use the old definition, i.e an array of them.

No, you can use a macro which makes your structs slower but changeable. There are several Julia packages offering this functionality.

my go-to strategy is to just make a module and write code there, and then if I need to change a struct definition, I just change the name of the module and re-run it.

ReplMaker.jl can make this process more convenient, but in version 1.9, there’ll be a facility for that built into the regular REPL.

Reply on news.ycombinator.com