Solving First Order Differential Equations with Julia | Hacker News Reader

Solving First Order Differential Equations with Julia

ritog.github.io

125 points__rito__39 comments

The first example is also the very first example in the docs[1], but rather f!(du,u,p,t) that modifies du, it passes f(u,p,t) that returns du. Searching the help for ODEProblem() function, it's explained[2] that the first way is more memory-efficient, but if mutation is not allowed, the second way is more suitable. It will be of interest to write a follow-up post elaborating on this part.

[1]: https://docs.sciml.ai/DiffEqDocs/stable/examples/classical_p... [2]: https://docs.sciml.ai/DiffEqDocs/stable/types/ode_types/#Sci...

I think I had volunteered to do something similar to this - let me check

Author here.

Yes, f(u,p,t) returning du is the suggested way to do things, but this pattern of updating du through in-place updating is more memory efficient when a system of equations (say 10-100) is involved.

The suggested way, when we don't need to think about memory efficiency, is easier to write and reason about.

I have plans to write more on this topic, and make it to a series. After I finish that, if there is enough interest, I will try and do a benchmark of mutating versus non-mutating styles with varying numbers of equations in a system of DEs.

Thanks for reading the article and providing your opinion.

X^2 - 3x - 18 =0
[...]

This is how the equation is created: x(x+3) = 18

wat

Maybe don't do very fast recaps if you're not going to proofread them. Incidentally I assume the formulae in this article were done with MathJax or its Julia equivalent, they render great but can't be copied from the text.

Overall a good article (and a great ad for Julia) but stumbling blocks like the one above ensure some readers won't make it any farther.

Is the issue here quadratic factorization is not obvious or that the sign is wrong?

Note that x(x+3) = 18 and x(x-3) = 18 are valid models of the same problem. One finds the longer side in terms of the shorter, or vice versa

Looks like proofreading (sign being wrong) to me.

Yes, it is absolutely that.

I have corrected it, and it should be okay now.

Both variants (+3 & -3) have the same shape, and so do their derivatives, but they do land in different places on the X axis. Is that irrelevant?

> "Note that x(x+3) = 18 and x(x-3) = 18 are valid models of the same problem. One finds the longer side in terms of the shorter, or vice versa"

That is the reason I made the mistake initially. They both were correct in my mind.

But that's no excuse for the mistake, and I made a correction.

Well if they made it that far, they also read that this article is for "people who are already familiar with Differential Equations from Mathematics." I think if you're already familiar, a single wrong sign is not something thats going to be particularly hindering. It's more just something to jump on if you are, for some reason, itching to criticize it.

It was my fault, and was created by a moment of absent-mindedness.

They are both correct individually, but definitely wrong when read together. I corrected it.

Thanks for reading the article and pointing out the obvious mistake. I will proofread more carefully and use a spell checker from the next time. There were some typos, too, that I corrected since then. Like 'langauge', 'equaion', etc.

I wrote the article in Jupyter Lab with a Julia kernel. I just wrote LaTeX in Markdown cells. I rendered the article with quarto and quarto is also how the blog is created. I am ignorant about how it renders LaTeX in webpages. Pandoc and MathJax seems to be involved. I am not sure.

> "Overall a good article (and a great ad for Julia) but stumbling blocks like the one above ensure some readers won't make it any farther."

Thank you for your compliment. I absolutely don't want people not making it any farther due to a small mistake I made. I will be more mindful about it from the next time.

doggo dot jl mention!

Thanks for the article. It gave me a little bit of familiarity with the DiffEq side of Julia, but even more valuable to me was the comparison between ordinary equations and diffeqs:

> In normal equations, we solve the equation to find out values of variables previously unknown to us. In case of Differential Equations, we solve to find out functions previously unknown to us.

That's a simple, easy to remember nugget of information that gives me some context to this whole field.

I really wish that somebody told me this when I was in High School. In HS, DEs were just solving integrals with a simple step away around as far as the student is concerned. I learned about them much more deeply when in college, and later independently.

As you know this now- if you come across a young person studying DEs, feel free to share this line with him/her.

Thanks for reading the article and sharing your opinion.

I didn't find a link to the package in the article. It seems to be a quite capable package, and the given two examples that are trivial to solve analytically don't really do it justice.

I have added a link to the package now. The first mention of the package in the body now contains a link to the official page for the package.

> and the given two examples that are trivial to solve analytically don't really do it justice.

Yes, I know. I plan to write more in the future.

Cool, the link is helpful, and the package is pretty cool. I'd just add that the examples are simple and be done with it, you can revisit with more later. Keep writing :)

It’s probably the best library for DEs in any language. I’m pretty sure that DifferentialEquations.jl, by itself, is the reason many scientists and engineers are using Julia.

https://docs.sciml.ai/DiffEqDocs/stable/

I use Diffrax and my own stuff with Numba/CuPy JIT-ing. Can you compare the Python ecosystem for DEs with Julia according to your experience?

No, I’m not familiar with the Python ecosystem for DEs.

I didn't try it yet, but the docs looks great.

can anyone recommend a good textbook on differential equations?

Could you tell me a bit about your background, your level in mathematics, and the type of book you prefer? Do you enjoy highly rigorous, proof-heavy texts, or do you prefer books where the author explains concepts in a more conversational and detailed way—still rigorous, of course, but more wordy?

I have a CS degree but haven't done math in a few years. Last course I completed was an MITx - intro to probability. While I enjoyed its rigor and detail, I am looking for something practical without proofs.

Oh well you might really like this, which I think is amazing. Steve Brunton from U Washington has an incredible course on DEs and dynamical systems. It’s a playlist on youtube where he talks through everything on a lightboard in a really incredibly clear way, there are tons of great examples, and he illustrates all the models using python and matlab code. And then a bunch of notes, problem sets etc are available online at the link.

https://faculty.washington.edu/sbrunton/me564/

I second that.

Also recommend this course by Gilbert Strang @ MIT:

https://ocw.mit.edu/courses/res-18-009-learn-differential-eq...

https://www.youtube.com/playlist?list=PLUl4u3cNGP63oTpyxCMLK...

Brunton is one of the greatest unsung YouTube stars for sure.

There is a course [0] from MITx that should be right for you. I had started it once, and it was very well-designed. But it was a bit slow-paced for my taste and I did not finish it.

For textbooks, I can recommend you the fantastic book- Non-Linear Dynamics and Chaos by Steven Strogatz. It is fully concerned with chaos theory and applications of DEs.[1]

I recommended these to you since you are looking for "practical" stuff.

A lot of DE courses deal with DEs mechanically, i.e. "here's this method, here are some examples, and this is how you use the method to solve it". I don't really like this approach, and I have never needed them outside college. If you like this, then look no further than Paul's Online Notes [4].

Here is also something you could consider: "Differential Equations: From Calculus to Dynamical Systems: Second Edition by Virginia W. Noonburg" [2] I haven't read it in full, but AMS generally has good taste when it comes to authoring good textbooks.

I also second the suggestion to check out stuff from Steve Brunton. [3]

If you want something practical, you might have a field in mind. So, you could also choose to read a basic course, and then dive into the field itself by looking up "Differential Equations in X books" where X is Physics, Biology, Finance, etc. For the absolute basics, Khan Academy is also good. I also mentioned two in the original post. Riley, Hobson, Bence is better for your needs.

I have recently come across Differential Equations with Applications and Historical Notes By George F. Simmons [5]. It might also float your boat.

I am aware that I have given you too many choices. But learning DEs is a months-long endeavor and I want you to spend at least some hours choosing your textbook/course/hybreed before you start your journey. This is how I chose my books in college. Multiple teachers, seniors would suggest ~10 books on each topic, and I would go to the library and spend two full days skimming through all of them, and choose 1-2 books.

[0]: https://mitxonline.mit.edu/courses/course-v1:MITxT+18.03.1x

[1]: https://www.stevenstrogatz.com/books/nonlinear-dynamics-and-...

[2]: https://bookstore.ams.org/text-43

[3]: https://databookuw.com/

[4]: https://tutorial.math.lamar.edu/classes/de/de.aspx

[5]: https://www.routledge.com/Differential-Equations-with-Applic...

Thanks, just bought them. What about calculus refreshers any recommendation?

Since you are eyeing DEs, I would suggest not to spend too much time on Calculus.

I have two resources in mind: Calculus Made Easy (available freely on Project Gutenberg) by Thompson, and simply Khan Academy.

Paul Lamar has detailed notes on Calculus, too [0].

[0]: https://tutorial.math.lamar.edu/Classes/CalcI/CalcI.aspx

I wish more people did this (get the required context) before steaming in with recommendations tbh.

That said, I will steam in with some starting points that may be helpful.

1) An “anti-recommendation”: Probably don’t get Edwards and Penney unless you are forced to for some course. It’s _fine_ I guess - like you will learn from it, but it is staggeringly overpriced for what it is and there are enough typesetting errors and other little niggles that grate when a book is as expensive as that. The one good part about it is there are tons of problems but for many/most of them it just gives the answer not a full solution, so it’s not very helpful if you are stuck or your solution looks very different from theirs and you don’t know where to go from there.

2) If you want a free pdf or online resource, mathematics libretexts has “Differential equations for engineers” by Jiri Lebl, which is at least as good as Edwards and Penney and is free and you get the pdf if you want to download it https://math.libretexts.org/Bookshelves/Differential_Equatio...

3) Dover Books publish “Ordinary Differential Equations” by Tennenbaum, Morris and Pollard, which I don’t have personally but a lot of people recommend. It’s a Dover book which means it is cheap and some of the terminology and notation is probably a little bit old-fashioned but it’s going to be a lot cheaper than Edwards and Penney if you want a physical book and as I say a lot of people recommend it.

So to complement what the parent said, one approach if you’re not sure what type of book you prefer, is check out Lebl (because it’s online and free so easy to dip into) and then you can explore from there.

But don’t get Edwards and Penney. I got a cheap second-hand copy and I still think I probably overpaid.

I would add get a CAS. You could use wxmaxima (which is OSS) or get Mathematica or something if you can get a cheap student license. It’s going to help a lot to develop intuition by allowing you to plot direction fields etc much more easily as well as doing some of the heavy lifting of verifying solutions etc (although you really need to do that a bunch yourself so you get good at it).

Here's an interesting book to consider:

https://www.amazon.com/Advanced-Engineering-Mathematics-Math...

https://www.amazon.in/Advanced-Engineering-Mathematics-Mathu...

https://khannapublishers.in/index.php?route=product/product&...

(This sells in India for a mere US $4. Costlier elsewhere.)

Pros:

* This has the most exhaustive catalog of differential equations and methods to solving them that I have ever seen. An example would be the discussion on non-homogenous differential equations that I have never seen anywhere else at all.

* While the subject matter is not easy, the book is complete in itself. A reader would not be led into a "dependency hell" or worse a cyclic dependencies of other materials to read.

Cons:

* Very low quality typesetting, print and paper quality. I wish a improved edition could be in the pipeline, however, the book is rather out of print.

* The book is not focused on differential equations. The other material covered is however a bonus.

Note: The authors were professors at my engineering college. Am however not associated with them in any way and I have no vested interest in recommending the book.

I'd consider Advanced Engineering Mathematics by Zill, 6th Edition which is available used for as little as 25 USD. There is also a print solution manual for this book which is great for self study.

Reply on news.ycombinator.com