Comment by the__alchemist | Hacker News Reader

Comment by the__alchemist | Hacker News Reader

Parent Full thread

I think the trouble comes when moving beyond numpy. I'm working on understanding chemistry more, and how to approximate wavefunctions for multi-particle systems. I wrote a Python script a few weeks ago that compares measured vs calculated psi'', to assess how good a trial WF is. Plotting 2/3 of the dimensions using a Matplotlib surface plot.

The surface plot brings my computer to a crawl. And, I'm not sure how to make it interactive. I gave up on it a few days ago.

Could have tried a Rust module, but ended up throwing together a custom Rust plotter in a few hours, with translated Python code. (I already have a basic WGPU-based rendering engine, with EGUI for UI). It runs much faster. They're both imperitive langs, so you can copy and paste, replacing `**` with `pow()`, `np.exp(x)` with `x.exp()` etc. And indexing with x[i][j][k] instead of vectorized numpy ops.

Ie the program compiles and runs in release mode faster than Python/numpy could perform the calcs. And the 3D graphics are smooth instead of a slideshow. And, I can make it real-time interactive since I'm using a low-level lang that integrates with GPU APIs directly. I'm not sure how feasible that would be in Python.

What did you use for your plotter? I'm in science as well and I'm shopping around for a good Rust library to use for just throwing together a quick plot of some simulation data. Currently I generate my data in Rust, write it to a numpy format, and look at it later with matplotlib, but as you've said, 3D plots usually end up looking like a slideshow.

I made my own, using WGPU and this basic graphics engine: https://github.com/David-OConnor/graphics_wgpu

Note that there's currently no docs or template/examples, and I'm rapidly breaking the API.

The surface plots are just meshes of a grid divided into triangles. When I need to manipulate them, I re-gen the meshes from a nested 3D array.

So, not a plotting lib at all; a flexible 3d and UI lib. Could probably made usable by others with a basic example of how you interact with the render and UI.

Have you tried the Plotter lib? I haven't used it, but looks nice from a skim. https://plotters-rs.github.io/book/basic/draw_3d_plots.html

Wow, very cool. Personally I'm not really interested in trying to go through building something like what you've done for myself, haha.

I've come across Plotters before but last time I checked, the documentation was pretty sparse and I didn't really relish the thought of trying to parse though and tweak the examples at that time. It looks like they're gradually filling out the docs now, though, so maybe I'll give it another go.

Rather than using matplotlib, you could try either pygfx (https://github.com/pygfx/pygfx) or fastplotlib (https://github.com/kushalkolar/fastplotlib) to make higher performance graphics using Python.

However, it won't solve your problem of Python not being fast enough doing the calculations.

Have a look at Julia. Julia is fast, and in addition its differential equations lib is possibly the best in the world.

I've used it in the past, but not recently. I remember that Diffeq lib, and it being best-in-class! Does it work for 3D (PDEs?) I remember having trouble applying it to that domain earlier, although it was outstanding for ODEs.

I think the big issue with Julia here is (other than the improving(?) JIT slowness (IIRC it was faster to compile and run a Rust program than JIT a Julia script), is I'm not sure how I'd interface with the GPU for graphics, compute shaders, and a GUI.

Julia's mathematical syntax is best-in-class; I wish more langs used something like that.

There are some nice tools for 3D PDEs which connect to DiffEq like GridAP (https://docs.sciml.ai/Gridap/stable/) and Ferrite (https://docs.sciml.ai/Ferrite/stable/). PDE tooling is where focus has been moving to as things evolve.

As for JIT, just today there was a PR that was merged that makes Julia cache and reuse binaries of packages (https://github.com/JuliaLang/julia/pull/47184). It won't be out until the next release of Julia, but it's a pretty major improvement to not JIT fully inferred package calls.

Great news, and TY for all your passion and work on the diffeq lib and Julia in general!

Reply on news.ycombinator.com