This completely contradicts my experience using LLMs, indispensable in my workflow for generating Julia code, a young language with little adoption outside its HPC niche. Furthermore, I use Julia for cryptographic software development, which is a niche in itself, and hence, there is no broad learning set from which LLMs could autocomplete.
One of my most impressive uses is for performance optimisation. Claude implemented efficient byte conversion to bigint, and I pasted Julia’s bignum standard base library file, `gmp.jl` to figure out how to do so [1]. Then, a Jacobi symbol calculation function was completely optimised with Calude, and I interactively provided feedback on the functions, correctness, and profiler information on which lines most of the time is spent [2].
Additionally, the OpenSSL wrapper for elliptic curve operations would be beyond my expertise if I hadn’t inquired with Claude about interfacing the OpenSSL_jll library for curve addition and multiplication. This led to the generation of 200 lines of Julia code, which I could execute in the REPL to explore the entire problem space [3]. These pieces were essential to get competitive performance for the ShuffleProofs package [4], where benchmark figures were also asked to be made by Claude with `Luxor`, which is instead a low-level library lovely in itself.
I would be unable to accomplish those improvements alone and would be highly reluctant to learn what a bignum limb is. It would be nearly impossible to implement those optimisations in such a short time while retaining the big picture of the ecosystem I maintain. I also used to have good experiences with ChatGPT at the beginning, but it silently became more dogmatic and lazy over time and somehow lost its ability to apply analogies between `C`, `Java` or `python` ecosystems.
To use LLMs effectively, one needs to interact with them appropriately and be aware of the biases one makes in the questions. Knowing when to make a gentle nudge and when to contradict is a vital skill to have there to get good answers.
[1]: https://github.com/PeaceFounder/CryptoGroups.jl/blob/0f6b4e2...
[2]: https://github.com/PeaceFounder/CryptoGroups.jl/blob/0f6b4e2...
[3]: https://github.com/PeaceFounder/OpenSSLGroups.jl
[4]: https://github.com/PeaceFounder/ShuffleProofs.jl