You can now simulate science in ways like never before. Today, the median scientist can easily rent a cluster of hundreds of nodes for a few hundred dollars an hour. It is increasingly the case that you can actually simulate entire products in silico before you do anything in the lab. SciML is a large part of that story because we are able to use ML to approximate science and speed it up even more.
I like to think about it as follows - 10x faster CPUs, 100x from GPUs when possible, 100x from ML when possible, 100x through easier access to parallel computing on cloud. So your best case speedup compared to a decade ago is easily 10^7x. Because of this huge space for improvement, we can easily find 1000x improvements in so many cases.
And this is what we as software engineers can do to change the world - by simulating science, building new batteries, designing new drugs, solving power infrastructure, getting climate right and its impact on our cities, food production, and so on and so forth.
Bret Victor captures this really well in his essay: http://worrydream.com/ClimateChange/ and at Julia Computing, we are doing a lot of what it outlines, and really grateful that ARPA-e and DARPA are funding all this hard science and improvements to Julia and its ecosystem.
I mean, it should be clear Julia is a much higher-performance method of crunching data than Python, which is unquestionably the king of the data science pile, and therefore it factors into performance & cost.
Scientific computing is chugging along about the same as it always has, quantum computers aren't really that relevant yet.
> I don't think the programming language makes that big of a difference, ultimately.
I can see how you might think this, but it's really ahistorical.
Scientific computing has always been a niche because of the range of skills needed. To have any real success at it as a team you needed to be a good enough at numerical analysis to understand the implementation, a good enough programmers to write something like production code (e.g. not your typical lab code) and good enough at the science to do the right project.
In the old days you were basically looking for one person who could do all of this, and in Fortran 77. You can carve off the last requirement if you only work on tools for other people, but that still leaves you with two domains.
Fortran 77 basically limited the scope of project that was reasonable. Things like matlab essentially started as wrappers on good libraries (in F77) so that people could get some work done without spending all their time fighting that complexity. This had a massive impact on productivity globally.
Introduction of things like c++ allowed more complex programs to be built for good or ill (also lead to improvements in fortran) but a lot of the same problems remain in terms of managing the complexity.
Later people added enough numerical libraries to python to get real work done, and that started to eclipse matlab at least in some specific domains (mainly because it's free and open).
Neither matlab or python are particularly good languages for scientific programming, but they are accessible - a gazillion grad students shoot themselves in the foot less in python than they would in fortran or c++, and iterate much faster.
In some ways systems like this have impact because they have reduced the necessary skill level across domains. There is always going to be room at the margins for a polymath but a lot of people who aren't can get things done much more easily now than a few decades ago. Now you may argue that nobody "does" scientific programming in python but it's a bit of a semantic flip, the core algorithms are all in c or something but depending on domain you may mostly be using python wrappers to access them.
Julia is an attempt (not the first one) to define a language that is both approachable and interactive (important) but also well designed for numerics etc. It's a very interesting project for that reason.
I've obviously skipped a lot of important stuff, but the impact of languages and particularly their accessibility has been really significant, especially when we get past scientific programming for it's own sake, and into real applications.
2) You are right the pay is worse in the public / government side. I will say - you might not believe me now, but it really is true that once you make enough money to pay all your bills, suddenly more money is less motivating than the ability to work on interesting problems.
What I just said above supposes that most lab work is interesting, which is not not the case, but I still think it's better than the average SWE's workload at your average tech company, just based on my personal experience having done both at different times in my career.
You'll really have to think about your priorities, but it sounds like you have a bit of time to do that.
Probably the worst case is "support programmer for a research lab", some people do well there if they love the lab and the work but it tends to combine poor pay with extremely limited options for professional growth.