Julia receives DARPA award to accelerate electronics simulation
juliacomputing.com
juliacomputing.com
My first academic paper (https://www.oxinabox.net/publications/White2015PsoTransistor...) was based on doing a search over parameters for the transistors being used in a circuit. The way that whole thing work was by string mangling the netlist file to update the parameters. Triggering the simulator on the commandline. Getting it to output a CSV of the signals. Parsing that CSV, and detecting edges and then measuring timing that way. Then throwing it at a particle swarm optimizer to fine a new one. That sucks from a user experience. It's a super cool way to solve problems though. that would be so easy with a real API.
Further, with julia being a differnetiable programming language, rather than having to use a particular swarm optimizer to search, I could have been differentiating it, and using some sophisticated gradient descent like LBGFS etc.
I hope that some of the general tools like this will be an outcome of this project.
It was a purely academic work for a masters level CS unit. I had just finished a masters level electrical engineering unit on silicon circuit design. Where the final project was to design an adder that minimized silicon used (and thus cost) while also being faster enough. And the hard bit is that you want big thick doped regions for high conductivity, but also the bigger the area the more paracidic capacitance. And so there are some tricks the to find good sizes, like progressive sizing and stuff. But afaik there is no actual answer, at least not one we ever learned. So a lot of trial and error went into it. It was a hard project.
And so then I did this CS unit where the project was "Do something interesting with a particle swarm optimizer". And i was like "lets solve this". and once I saw the results, i was like "this is actually really good", and so the lecturer and I wrote a paper about it.
It is a real problem. Minimizing silicon area subject to speed. I bet the big integrated designers have there own tricks for it that i don't know about. To do it really well you need to miminal the real area so also need to solve layout (which is a cool cutting and packing problem). (and ther are also nth order effects, like running traces over things can cause slow downs, because electromagnetism reasons) I bet a bunch of folk on HN know this problem much better than i do though. probably something bad in my solution, but i think it illustrates the utility
Now that everyone is using finfet processes, the layout part is pretty easy to solve because transistor widths have to be a certain number of fins and the layout is extremely regular.
One thing your analysis didn't include, which actually ends up being quite significant, is the extra capacitance caused by the wires between transistors. This changes the sizing requirements substantially.
I've done some custom logic cell design, and I always had to use a lot of trial and error, though generally I was concerned more with speed than area. I'm not sure exactly what the development process is at my current employer, but it seems like its a lot of manual work. I'm guessing they set area targets based on experience and attempt to maximize speed where possible.
Ultimately, everything gets placed and routed by a computer anyways!
Indeed, I am actually surprised the paper doesn't include something like _"This is inline with the well known result for progressive sizing [cites textbook]"_. It was my first paper, i was worse at writing things then. :-D
> One thing your analysis didn't include, which actually ends up being quite significant, is the extra capacitance caused by the wires between transistors. This changes the sizing requirements substantially.
Good point. And not easy to model in a SPICE style simulator. I guess one could maybe introduce explict capacitors and them compute capacitiances by making some assimptions about layout.
That is, in fact, exactly what we do! I think it would be pretty straight forward for your large buffer example - you can model it as a fixed capacitance at each output which corresponds to the routing between inverters, which would be the same for all sizes, plus some scaling capacitance that relates to the size of the transistor itself, which you already have.
The adder would be trickier, for sure. Regardless, in my experience, just adding a reasonable estimate is good enough to get you close in terms of sizing in schematics, then you fine tune the layout.
The EDA tools have no ecosystem you can hook into and they don't really care about the user trying to put the simulators into a flow to solve their problem. It is a bunch of point tools each with their own embedded interpreter that don't play together. I sure hope there is a plan to create a better set of tools so I can write custom netlist checks and do something novel (like get derivatives out of the simulator) and in-memory (no slow disks) and run custom Julia checks during simulation. Julia is a much better match because running Python or MATLAB code within the simulator is way too slow. I'll keep watch for sure.
^ stands for bitwise XOR: so [expr {-2^2}] results in -4
* stands for exponentiation: so [expr {-2*2}] results in 4
Both seem correct to me, taking into account how integers are represented in binary (two's complement for the negative ones).
With regards to debugging dynamic programming languages, it is different as compared to their static counterparts, since much is delayed to happen at runtime (as opposed to at compilation time). But it also opens up possibilities (like introspection, ability to intervene in the scripts while they run, ...). It requires a different mindset.
** stands for exponentiation: so [expr {-2**2}] results in 4
its early days but you can start playing with our beta
There needs to be an open initiative between semi companies to create a standard simulation environment -- with compilers, unit-test frameworks, and all sorts of simulation (gate-level, analog/mixed signal, emulation, etc). Hell, just give me a free IDE plugin for SystemVerilog that actually works.
This lack of a standard seems to me like the critical path in hardware design. I'm trying to support projects to fix this like SVLS (A language server for SystemVerilog: https://github.com/dalance/svls) but these are all hard problems to solve. This industry is relatively niche and doesn't seem to have many engineers interested in FOSS.
With that out of the way...
They could argue that since it was a tool that was made to help with his job at at the company, then it's internally developed. If there are no clear grounds or easily presentable evidence (even if there was!), he's out in the wild with a liability.
The only place where I can see him in the clear is if they had a repo going back to before the work on the company and they could prove the tool was a personal project, and unrelated to the job. Even then, Company could still sue and burden them with legal fees/process until they tire him out and BINGO, now they own their IP.
The alternative could be starting a new project, completely open source from the start (probably with one of the more liberal licenses) and get crowdfunding to develop and maintain it. Assuming they are interested in doing that, of course.
I mean ... it's a really nice tool, they are giving away from free anyways ... if they OpenSourced it, the kind of feature you describe here (and which I've been sorely missing as well) would be a no-brainer to implement.
Not to mention all the other enhancements the community could and would bring to it.
From your other comments, it seems that the general principle is to create a simple model that captures most of the behavior then apply corrections on top of that.
Can you elaborate on how the model is trained? Is it just from netlist/models? Or are you running a regular transient simulation? Or do you need a special transient setup?
What happens when your circuit's operating point varies wildly during operation? Presumably you'd have to train a larger model over the entire range of operating points but it seems to me that it would require extensive simulation to just collect the operating point data, which somewhat defeats the purpose. You've also got simulation corners, where you'd have another massive set of permutations to generate models for...
My other big concern is accuracy. Chris commented that you're achieving these speed ups at 99% accuracy. Does that mean your results are within 1% of the "full" simulation? Or that the intrinsic simulation error is 1%? For the former, that result is extremely dependent on the particular simulation and desired output. I'm curious if you have preliminary results on real circuits? If it's the latter... That's not enough. Even for mediocre circuits you're looking at >60dB relative accuracy requirements. Potentially >100dB for some high performance applications. The default relative tolerance is 1e-3 and we often reduce that by an order of magnitude or two...
There are certainly mixed signal simulations where we care more about functionality than performance, so it could help there. But as a matter of practice we usually already have verified verilog models for all analog blocks anyways.
And lastly, I'm curious how much does this have to do specifically with Julia the programming language? Is it just an "all part of the family" kind of thing? There's no claim that something intrinsic in the language is providing speed up, right? It's just a conveniently easy way to implement?
It's not applying corrections as in doing a taylor expansion and then adding terms to it, it's basically a projection of the systems. The basic details are in the CTESN paper, though I think there's improvements that haven't been published yet to make it work in this domain. Another point to note is that sometimes the "simpler" model actually as more equations, but the equations are of a form that is much faster to simulate, because you can relax some stiffness in the original set of equations.
> Can you elaborate on how the model is trained? Is it just from netlist/models? Or are you running a regular transient simulation? Or do you need a special transient setup?
The details are complicated, but the basic approach is the usual ML thing: You pick some highly parameterizable model and projection, and then take gradients to tweak the parameters until the projection reproduces what you want, except in this case it's obviously a continuous time problem and you don't need to feed in data, because you can just use the original simulation in your loss function. For some more advanced techniques you do benefit from having the whole simulator be differentiable even on the baseline, but conceptually it's not required and you could do it blackbox with a regular transient simulator.
> What happens when your circuit's operating point varies wildly during operation?
You can choose what parameters to fix and which to keep variable over the training process. It is probably true that in some (e.g. chaotic) models this means that the surrogate generation is too hard and will fail to converge for reasonable sizes of the model. We don't have enough experience yet to give you a clear answer of when this happens or not - figuring that out is part of the research effort here.
> My other big concern is accuracy. Chris commented that you're achieving these speed ups at 99% accuracy. Does that mean your results are within 1% of the "full" simulation?
I'm not sure I quite understand the distinction that you're drawing and measuring errors in a sensible way here is actually somewhat non-trivial. I believe the error rates Chris quoted are from a smooth distance metric between the signal generated by a full simulation and those generated by the surrogate.
These tradeoffs are tunable of course, so for a particular application you can decide how much accuracy loss you can live with. Of course, there will be infeasible points for this technique but hopefully it's useful. I can't speak for DARPA, but if you look at the original call for proposals, you will see that they were asking for accuracy targets in the single-digit percent, so we think at those targets the 1000% speed-up is more than achievable.
> There are certainly mixed signal simulations where we care more about functionality than performance, so it could help there. But as a matter of practice we usually already have verified verilog models for all analog blocks anyways.
One way to think of this is as automatic generation of those verilog models from the netlist for the mixed signal use case. It is certainly still at the research phase though - where it's applicable will highly depend on what the performance/accuracy trade-offs look like and how expensive the thing is to train.
> And lastly, I'm curious how much does this have to do specifically with Julia the programming language? Is it just an "all part of the family" kind of thing? There's no claim that something intrinsic in the language is providing speed up, right? It's just a conveniently easy way to implement?
We've spent many millions of dollars building some extremely sophisticated modeling & simulation tooling in Julia, most of it open source, so we get to piggy back on that here to get a very modern simulator with all kinds of bells and whistles "for free", including the surrogatization capabilities. We are expecting speedups on real-world problems just by using this framework for baseline simulation also, but I don't have data on that yet, so I don't have any concrete claims to make. As for the question whether such a speedup is intrinsic to the language, as always the answer is yes and no. Julia's design makes it extremely easy to write very high performance code. Both of those things are important. Being easy to use, allows you to do go further down the algorithmic rabbit hole before you start hitting the complexity limit :).
Are you able to share what some of those are?
> You can choose what parameters to fix and which to keep variable over the training process
Can you explain more about the parametrization? Do your parameters correspond 1-to-1 with schematic parameters like transistor sizes or resistance/capaciance values? Or internal transistor model parameters? Or are they more abstract mathematical parameters?
> I'm not sure I quite understand the distinction that you're drawing and measuring errors in a sensible way here is actually somewhat non-trivial.
Circuit simulator vendors often market their accuracy in terms of "% SPICE Accuracy", and what they mean is if you run a simulation and measure some parameters (usually something like RMS noise voltage or signal-to-noise ratio), then those measurement results will be within 1% of what the measurement results you'd get from running the same simulation with full-accuracy SPICE.
The other way of measuring simulator accuracy is in terms of dynamic range. For example if I have a noisy sine generator circuit where the signal has rms amplitude 1, and the noise is 1e-4 rms, I need to make sure the numerical noise of the simulator is much less than 1e-4.
The first is sort of relevant in your case as a comparison between surrogate and full simulation. The second is an absolute measurement of a single simulator's (or surrogate's) accuracy.
> I believe the error rates Chris quoted are from a smooth distance metric between the signal generated by a full simulation and those generated by the surrogate.
This is interesting because it makes sense as an application-independent metric for measuring the accuracy of your surrogate. It's not immediately clear to me how this would translate to circuit performance accuracy in all cases. However, in one specific case - a Digital to Analog Converter - that 1% smooth distance error could be catastrophic for some performance metrics depending on how it behaves.
That might be an interesting thing for you to consider investigating through the course of your research.
> We are expecting speedups on real-world problems just by using this framework for baseline simulation
Very cool, and I'm looking forward to seeing some data there.
One last question - why did you choose this particular DARPA project? Was it any specific interest/relationship with the circuit design industry? Or did it just happen to be a very cool application of CTESN?
The papers are being written, so should be public in a few months, but I can't go into detail quite yet.
> Can you explain more about the parametrization? Do your parameters correspond 1-to-1 with schematic parameters like transistor sizes or resistance/capaciance values? Or internal transistor model parameters? Or are they more abstract mathematical parameters?
All of the above. We have a fully symbolic representation of the circuit, so any part of it can be replaced by a parameter. Transistor sizes and device values are a natural thing to do, but the system is generic.
> % SPICE Accuracy"
Heh, we've actually found some SPICE implementations to have significant numerical issues when compared to our baseline simulator, which has some fancier integrators - I suppose that would make us worse on this metric ;).
> That might be an interesting thing for you to consider investigating through the course of your research.
Yup, characterizing error trade offs is one of the primary outcomes of this research.
> One last question - why did you choose this particular DARPA project? Was it any specific interest/relationship with the circuit design industry? Or did it just happen to be a very cool application of CTESN?
Circuit design was my first job, and I have a second desk with a soldering iron and test equipment, though it's collecting dust ;). I have a bit of a hypothesis that - between open source tooling becoming more mature, and more people getting into the chip design space for custom accelerators - we're on the cusp of a major overhaul in EDA tooling. I thought we'd be good at it, so I was advocating internally for us to start up a team in the domain. We have a bit of an "if you can get it funded it, you can do it" attitude to those sorts of things, so I was trying to find the money to jumpstart it, and this is that.
Hah. That's not surprising. Our vendors generally mean spectre when they talk about full accuracy, but I have a laundry list of grievances when it comes to spectre and its accuracy settings... What are you using as your baseline?
> Circuit design was my first job
Glad to hear. This industry needs a lot of modernization.
> I have a bit of a hypothesis that - between open source tooling becoming more mature, and more people getting into the chip design space for custom accelerators - we're on the cusp of a major overhaul in EDA tooling
I hope you're right. The current state of circuit EDA tooling is abysmal. I think on the analog/RF side, the ecosystem is so ancient and entrenched that it will take a herculean effort to make any real strides, especially because the golden age of analog/RF semi startups is over. But digital design is very much becoming open source, as you mentioned, and maybe that will eventually bleed over.
I wish all types of circuit design were more accessible to the world - lots of emerging economies could use it, but the up front cost is just so high. There's been a massive surge of software dev recently in the Middle East and Africa, and hopefully hardware dev follows suit.
Julia would of course be ideal for this, and I very much like where you project is going!
Someone made a blog post recently comparing the time to simulate with LTSpice vs writng and solving the system in Julia https://klaff.github.io/LTSvDEQ_1.jl.html this is a very simple circuit, and they still got a 100x speed up. Sure that is neglecting the time it takes to actually extract the differnetial equetion from the circuit. But from what i hear that kind of thing is something this DARPA project will be working on. And sure LTSpice isn't state of the art. But still I find this indicative and promising.
1. It’s free unlike Altium addons or orcad p-spice
2. Graphical, I’m happy to code things but code literacy varies widely among EEs. Much easier to share results when it looks like a schematic
3. Good enough component library. The time spent finding and inputting component parameters are gonna be way bigger than any savings on the actual computation
I mostly work on small embedded systems boards and use simulation to probe behavior of analog sub systems I’m concerned about, rather than simulating the whole board. Maybe more complex designs get more like CFD models where computation time is measured in hours or days. Would love to see someone use this as a backend for an alternative to the major spice programs, LTSpice UI isn’t exactly pleasant, and is unusable on Mac so it wouldn’t take a whole lot to get me to switch.
I think speed is very much a question of what kind of thing you are doing. I agree it often doesn't matter. and without your 3 points, it certainly doesn't matter
I only simulate one pulse (about 3 ms), but the simulation takes minutes to resolve the inrush. Tuning circuit impedances to match measurement is a real pain. At this point I'm just going to take many more direct measurements. If it was quick I would have written a script to scan through unknown parameters to maximize correlation with measurements, but that isn't reasonable when the simulation fails after several minutes of trying for most values.
Of course, if we got enough money to actually employ a developer, that would be amazing. It's just not very realistic. Furthermore, having BioJulia be developed by working scientists has its advantages.
If you, or anyone else, is interested in BioJulia, do think about making a contribution to your favorite package, it would be very welcome. Developing in Julia is extremely satisfying, as you get so much bang for your buck, while still being able to create highly efficient code.
Usually, the best packages come about when people are motivated to creating something specific, for example if they think the status quo in some domain is not good enough.
I'm sure we can dig up a handful of old, badly maintained projects that could use some love. Off the top of my head, it would be nice to have
* Micro-optimized our smith-waterman algorithm. That's probably fairly easy to get started with if you're not a bio person
* A number of our parsers have not been properly maintained. We use finite state automata https://github.com/BioJulia/Automa.jl to create parsers. That's for more advanced users
* We need to consolidate our scattered sources of k-mer analysis code. Another developer is re-writing our k-mer iterator protocol, but we need a big toolbox for k-mer counting, minhashing, calculating k-mer spectra etc. That's also very computer-sciency and no so much biological
Feel free to get in touch on the Julia Slack, or send me an email :)
In my broader experience, almost no bioinformaticians use Julia. I think we, as a field, are more conservative than e.g. physicists when it comes to technology. My old institute taught me Perl as the lingua franca of bioinformatics as late as 2015 (but switched to Python the year after).
I think we, as a field, have been consistently fairly poor at choosing our programming tools. Old bioinfo scripts are cluttered mess of write-only spaghetti-Perl. Most bioinformaticians I know don't use Biopython or Bioperl or anything similar, but rather creates new programs or packages by either re-implementing the basics from scratch, or by duct-taping together static binaries and/or scripts through shell commands.
We will never get rid of having to use static binaries or external scripts, but I think BioJulia at least have a decent chance of stopping people from re-implementing the basics again and again, and providing a central "platform" that various external scripts communicate through (e.g. an old Perl script may produce DNA as a FASTA file, when can then be fed into BioJulia). The main issue is to have bioinformaticians understand the current situation is problematic.
It doesn't take much to have a big impact, I think. If we had an ecosystem of the most basic data types (biosequences, kmers, phylogenetic trees and protein structures), a collection of well-known fundamental functions to operate on them, and parsers for the 20 most common formats, we would already have a very compelling ecosystem.
SID was an artificial intelligence rule-based system and expert system with over 1000 hand-written rules. In addition to logic gate creation, SID took the design to the wiring level, allocating loads to nets and providing parameters for place and route CAD tools. As the program ran, it generated and expanded its own rule-base to 384,000 low-level rules.[19][20] A complete synthesis run for the VAX 9000 took 3 hours.
Initially it was somewhat controversial but was accepted in order to reduce the overall VAX 9000 project budget. Some engineers refused to use it. Others compared their own gate-level designs to those created by SID, eventually accepting SID for the gate-level design job. Since SID rules were written by expert logic designers and with input from the best designers on the team, excellent results were achieved. As the project progressed and new rules were written, SID-generated results became equal to or better than manual results for both area and timing. For example, SID produced a 64-bit adder that was faster than the manually-designed one. Manually-designed areas averaged 1 bug per 200 gates, whereas SID-generated logic averaged 1 bug per 20,000 gates. After finding a bug, SID rules were corrected, resulting in 0 bugs on subsequent runs.[19] The SID-generated portion of the VAX 9000 was completed 2 years ahead of schedule, whereas other areas of the VAX 9000 development encountered implementation problems, resulting in a much delayed product release. Following the VAX 9000, SID was never used again. [Not sure why]
It was the last machine that DEC made that did not use a microprocessor.
The money is going to a company.
Electronics simulation is fascinating, especially given the AI models used to do this. Layout gets especially complex when rf-traces / layers have to be considered or when you want to have an arrangement of traces to high bandwidth components all be the same length. Interaction between multi-layer vias is also insane (the guy who built the UberTooth1 bluetooth hacking dongle has a great defcon talk on the subject).
The best analog I can come up with is the debate / discrepancies between the US and European weather simulation models. Fascinating space, I got to work a few feet away from the Julia team at the Harvard Launch Lab way back in the day at a college internship. Of the few interactions I had with their team they are great people and unbelievably brilliant. If any of you are reading this, my hat's off to your engineering abilities - I'm still impressed by the fact you guys identified an error in intel's x86 instruction sets and yeeted the issue in less than 24hrs.
Granted, I make a great living writing software and honestly have really benefitted from my 67% complete EE degree. Software ppl generally have zero idea how computers work / how to really leverage hardware bits to accelerate certain workloads. The ideal CS education for me is based in EE but also starts with both lisp and C. NOT Python. However, I was a horribly distracted student throughout college so I really should be the last person giving recs for coursework.
When people are at the very beginning of some path, they have almost no attachment to it and the smallest nudge one way or the other can change their course.
Think about how many people say, "If it wasn't for <random elementary school teacher> I would have never gotten into <field they became famous in>."
I don't work as an Electronics engineer any more but I still have a significant interest.
Also LT-Spice being native desktop is great. No way the lock it into the cloud now.
edit: similar in approach, not as a 1:1 mapping. Replace a deterministic model with a faster, slightly fuzzy one.
What's the alternative to using a NN for position evaluation? I can think of two:
1. Do minimax search until you have a winner or a stalemate. Then you have the exact value of the position. Well, this is the problem we're actually trying to solve to begin with, and it's also impractical to do for chess and any interesting game. This is what necessitates an approximation to position evaluation.
2. A human expert writes a position evaluation function. It determines a huge handful of features, a simple example being how many pieces I have - how many pieces you have, and some way to combine those features into a score.
In surrogate modeling, you can get ground-truth data to evaluate your approximation against. You're approximating another model that you can compute, it's just too slow for practical use.
In chess, we don't know THE position evaluation function. We can certainly get data about it, but we don't know it in the same way we know PDE models.
To be clear, I am not saying we _know_ the PDE models are accurate with respect to reality. That's the science part, to determine if the model arising from empirical evidence or first-principles that are themselves arising from empirical evidence, actually summarizes empirical evidence.
Shouldn't this be at around "at around 99% accuracy" or "within around 1% accuracy loss"?
1. projections / disregarding some dimension, followed by some model in this latent space, followed by an injection back into the native space. The model is simpler by virtue of operating in a lower dimensional space.
2. using a "reduced-order model" or "surrogate model", but still operating in the same input/output space. The model is simpler by virtue of, e.g. using fewer derivatives or delays.
And with 2, you can have have the simple model e.g. arise from first principles, and learn a _residual_ correction on top of it.
And it sounds like the comment contains a bit of both. Am I getting that right?
If you want to see next generation circuit simulation and automation that already working check out JITX. The simulator is using their advanced LB Stanza language which similar if not better than Julia[1]. This is the same team from Berkeley that proposed CHISEL and FIRRTL.
Other promising efforts on next generation digital and analog circuit design are from MLIR by LLVM team and LLHD from ETH Zurich [2][3].
[1]http://lbstanza.org/ [2]https://github.com/llvm/circt [3]http://llhd.io/
One additional point: Julia depends on OpenBLAS not because Fortran is faster, but because doing the architecture tuning for all supported architectures and sizes is a bit of a pain and for standard BLAS problems there's very little reason to switch. We do have pure Julia packages that outperform OpenBLAS, but nobody has gone through the effort of replacing the base usage of BLAS usage and completing the pure Julia packages to achieve 100% API coverage. There's just no good reason to as long as the vendor BLAS packages (or OpenBLAS) work fine.
Don't get me wrong I'm not dismissing your work, I'm just dismissal of the outrages 1000x claim, as they say "extraordinary claims require extraordinary evidence"(ECREE). I believe Chris Lattner and Patrick Li will probably cringe to hear the 1000x claim ;-). Like I've mentioned before, I love to be proven wrong.
Regarding OpenBLAS, it will be very good to have native Julia alternative as you've claimed. It is nothing new to be better than OpenBLAS since D language has done it with probably less than 10x (100x?) man power compared to Julia several years back[1].
[1]http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/...
https://github.com/JuliaLinearAlgebra/Octavian.jl/issues/24#...
but also on Intel:
https://chriselrod.github.io/PaddedMatrices.jl/dev/arches/ca...
I personally wouldn't spend too much time on BLAS-limited applications though, and this kind of circuit modeling is not one of them as I describe in another post. Also, it's 1000x at 99% accuracy: it's essentially a form of automated model order reduction which allows you to choose a tolerance and get more speedup matching the original circuit to the given tolerance.
https://github.com/YingboMa/RecursiveFactorization.jl#perfor...
And there are hooks in the differential equation solvers to not use OpenBLAS in many cases for this reason:
https://github.com/SciML/DiffEqBase.jl/blob/master/src/linea...
Instead what this comes out to is more of a deconstructed KLU, except instead of parsing to a single sparse linear solve you can do semi-independent nonlinear solves which are then spawning parallel jobs of small semi-dense linear solves which are handled by these pure Julia linear algebra libraries.
And that's only a small fraction of the details. But at the end of the day, if someone is thinking "BLAS", they are already about an order of magnitude behind on speed. The algorithms to do this effectively are much more complex than that.
Google has been doing electronic with ML for some time now : https://ai.googleblog.com/2021/02/machine-learning-for-compu...
Python and Tensorflow work just fine for that task, the speed comes from LLVM and dedicated hardware such as TPU. The language in which the high level ML network/algorithm is specified doesn't matter much.
It is always good to have several paths explored, but I'm skeptical on the Julia hype since its beginning.
https://github.com/ModiaSim/Modia.jl
The authors of that tool have a strong background in modeling and simulation of differential algebraic equations. Not so much in designing DSLs, though, so there maybe some technical oddities. But I expect the simulation aspect to be quite decent.
https://www.stochasticlifestyle.com/generalizing-automatic-d...
or it can work via a component-based modeling DSL:
https://people.inf.ethz.ch/fcellier/Res/Soft/Dymola_engl.htm...
We've had the benefit of learning from them and their presentations at JuliaCon and collaborating. More is coming. In the meanwhile - do see this JuliaCon talk by Hilding:
Do they want to generate efficient FPGA programs?
FPGAs are already digital... meaning that although they (FPGAs) have to be simulated when they are designed, their purpose is digital simulation. Programs are at their heart a discrete digital simplification of the underling electrical operation for the purpose of increasing reliability and allowing greater design complexity.
https://en.wikipedia.org/wiki/SPICE
Also see HSPICE, pSPICE, LTspice...
Also, why doesn't the funding go to more general linear algebra (and related) solver software, which can then be used in many physics packages?
2) You are right the pay is worse in the public / government side. I will say - you might not believe me now, but it really is true that once you make enough money to pay all your bills, suddenly more money is less motivating than the ability to work on interesting problems.
What I just said above supposes that most lab work is interesting, which is not not the case, but I still think it's better than the average SWE's workload at your average tech company, just based on my personal experience having done both at different times in my career.
You'll really have to think about your priorities, but it sounds like you have a bit of time to do that.
Probably the worst case is "support programmer for a research lab", some people do well there if they love the lab and the work but it tends to combine poor pay with extremely limited options for professional growth.
I mean, it should be clear Julia is a much higher-performance method of crunching data than Python, which is unquestionably the king of the data science pile, and therefore it factors into performance & cost.
You can now simulate science in ways like never before. Today, the median scientist can easily rent a cluster of hundreds of nodes for a few hundred dollars an hour. It is increasingly the case that you can actually simulate entire products in silico before you do anything in the lab. SciML is a large part of that story because we are able to use ML to approximate science and speed it up even more.
I like to think about it as follows - 10x faster CPUs, 100x from GPUs when possible, 100x from ML when possible, 100x through easier access to parallel computing on cloud. So your best case speedup compared to a decade ago is easily 10^7x. Because of this huge space for improvement, we can easily find 1000x improvements in so many cases.
And this is what we as software engineers can do to change the world - by simulating science, building new batteries, designing new drugs, solving power infrastructure, getting climate right and its impact on our cities, food production, and so on and so forth.
Bret Victor captures this really well in his essay: http://worrydream.com/ClimateChange/ and at Julia Computing, we are doing a lot of what it outlines, and really grateful that ARPA-e and DARPA are funding all this hard science and improvements to Julia and its ecosystem.
Scientific computing is chugging along about the same as it always has, quantum computers aren't really that relevant yet.
> I don't think the programming language makes that big of a difference, ultimately.
I can see how you might think this, but it's really ahistorical.
Scientific computing has always been a niche because of the range of skills needed. To have any real success at it as a team you needed to be a good enough at numerical analysis to understand the implementation, a good enough programmers to write something like production code (e.g. not your typical lab code) and good enough at the science to do the right project.
In the old days you were basically looking for one person who could do all of this, and in Fortran 77. You can carve off the last requirement if you only work on tools for other people, but that still leaves you with two domains.
Fortran 77 basically limited the scope of project that was reasonable. Things like matlab essentially started as wrappers on good libraries (in F77) so that people could get some work done without spending all their time fighting that complexity. This had a massive impact on productivity globally.
Introduction of things like c++ allowed more complex programs to be built for good or ill (also lead to improvements in fortran) but a lot of the same problems remain in terms of managing the complexity.
Later people added enough numerical libraries to python to get real work done, and that started to eclipse matlab at least in some specific domains (mainly because it's free and open).
Neither matlab or python are particularly good languages for scientific programming, but they are accessible - a gazillion grad students shoot themselves in the foot less in python than they would in fortran or c++, and iterate much faster.
In some ways systems like this have impact because they have reduced the necessary skill level across domains. There is always going to be room at the margins for a polymath but a lot of people who aren't can get things done much more easily now than a few decades ago. Now you may argue that nobody "does" scientific programming in python but it's a bit of a semantic flip, the core algorithms are all in c or something but depending on domain you may mostly be using python wrappers to access them.
Julia is an attempt (not the first one) to define a language that is both approachable and interactive (important) but also well designed for numerics etc. It's a very interesting project for that reason.
I've obviously skipped a lot of important stuff, but the impact of languages and particularly their accessibility has been really significant, especially when we get past scientific programming for it's own sake, and into real applications.
This might give a nice boost to adoption of Julia. And hopefully this spending trickles down to further improve the language itself.
---
Not saying you can't do this stuff in rust. I mean its turing complete.
But contrast the description of Rust: A language empowering everyone to build reliable and efficient software.
vs Julia Julia is a high-level, high-performance dynamic language for technical computing.
Julia is for doing "technical computing". Things like simulations, problems where you need to apply a bunch of math. Its easy to write math in julia. The syntax is designed for it (like look at code for manipulating matrixes). The semantics are designed for it (multiple dispatch is the thing you need for efficient linear algebra for example. Specializing on each combination of matrixes)