HNHacker News
TopNewBestAskShowJobs

cbkeller

1,804 karma · joined October 22, 2018

geologist - brenhinkeller.github.io
submissionscomments
cbkeller··on The Accelerating Adoption of Julia
Hah, you were not kidding about the names being unintuitive.

For anyone else interested there is a large list starting here [1]. The in-place versions will all end with `!`.

[1] https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/#Linea...

cbkeller··on The Accelerating Adoption of Julia
I understand you are frustrated, however, please remember

> Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."

> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.

> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.

https://news.ycombinator.com/newsguidelines.html

cbkeller··on Einstein's missed opportunity to rid us of 'spooky actions at a distance'
This is fairly far from my field, but as I understand it, that would be a hidden variable interpretation of QM [1], and specifically in that analogy a local hidden variable theory. That's what Einstein himself wanted.

There is a famous test, Bell's inequality [2], that specifically rules out local hidden variable interpretations of QM.

Nonlocal hidden variable interpretations, such as De Broglie - Bohm theory [3], are potentially still on the table, however.

It is somewhat ironic that Bell's theorem is sometimes presented in popular media as a general disproof of all hidden variable theories, in a context where locality is taken for granted -- because Bell himself seems to have been partial to nonlocal hidden variable theories. An article by the same Mermin mentioned in the OP is worth a read, on this subject [4].

[1] https://en.wikipedia.org/wiki/Hidden_variable_theory

[2] https://en.wikipedia.org/wiki/Bell%27s_theorem

[3] https://en.wikipedia.org/wiki/De_Broglie%E2%80%93Bohm_theory

[4] https://cqi.inf.usi.ch/qic/Mermin1993.pdf

cbkeller··on What happened to South America's missing mega-mammals?
They're soils, but they're special soils [1] that can harden to a brick-like consistency (called a "duricrust") upon being dried out and exposed to the air [2,3].

The unusual durability of such soils has made them a target for study in their own right, using cosmogenic isotopes to estimate erosion rates over Myr timescales (which turn out to be incredibly slow; 0.16–0.54 m/Myr) [4]

[1] https://www.cambridge.org/core/journals/clay-minerals/articl...

[2] https://en.wikipedia.org/wiki/Duricrust

[3] https://www.sciencedirect.com/science/article/pii/S016913681...

[4] https://www.sciencedirect.com/science/article/abs/pii/S00128...

cbkeller··on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R
> precompiled and even distributed

There is work on this. This can be done with https://julialang.github.io/PackageCompiler.jl/dev/, but so far only to a rather large binary / “bundle”

IIUC, In principle it should be possible to do much better if you knew for sure at compile time which methods you would need to dispatch to for the data you ultimately want to run on.

cbkeller··on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R
That’s why Pandas read_csv is what they benchmarked
cbkeller··on What happened to South America's missing mega-mammals?
Geologist here! A fun fact that has received some previous discussion [1] here:

Some of these same missing megafauna (likely giant ground sloths) dug an impressive network of several-meter-diameter tunnels through the unusually stable, iron-rich soils of southern Brazil, which survive to this day [2].

You can actually go and walk through these burrows, despite the fact that the species who dug them are long-extinct.

[1] https://news.ycombinator.com/item?id=14014049

[2] https://www.discovermagazine.com/planet-earth/get-lost-in-me...

cbkeller··on Nilang.jl – A Reversible Julia DSL
I was wondering about this latter point as well, and apparently this is exactly the goal if one could implement reversible computing at the hardware level [1]:

> One of the earliest proposals to reduce heat from lost bits was to make computer circuits reversible. A reversible circuit has exactly as many outputs as inputs, assigning one output pattern to every input pattern and vice versa. That means each input can be reconstructed from the output; no bits are lost, so reversible circuits will not give off heat from bit loss. Furthermore, reversible circuits can simulate standard logic functions and can therefore be used in any computer.

[1] https://www.americanscientist.org/article/computers-that-can...

cbkeller··on Nilang.jl – A Reversible Julia DSL
I think it's a variant of the latter -- IIUC no information can be "lost" in reversible computing, so the output of a reversible hash function might be both the hash and some number of other outputs, all of which you would have to know if you wanted to reverse the hash function.

It seems there is an analogy to be drawn to the way in which there can be no "waste heat" in a reversible thermodynamic process [1]. I thought this analogy might be a bit of a stretch at first, but looking into this a bit more it seems as though this is indeed exactly the idea with reversible computing, such that if a reversible computer could be implemented at the hardware level there would supposedly be significant energy efficiency gains on the table [2-4].

[1] https://en.wikipedia.org/wiki/Reversible_process_%28thermody...

[2] https://arxiv.org/abs/1702.08715

[3] https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/INC19-...

[4] https://spectrum.ieee.org/computing/hardware/the-future-of-c...

cbkeller··on Nilang.jl – A Reversible Julia DSL
I'm probably just missing something, but did you mean to say "fast derivatives of scalar functions" by any chance? I.e., that you can use the free inverses to get at the derivatives in some way?
cbkeller··on Javis v0.2 and the Future
For someone who's been following this only tangentially, how similar is the interface and implementation to what's used in Grant's https://github.com/3b1b/manim ?
cbkeller··on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R
Well, the promise of overhead-free multiple dispatch is that you don't have to choose between a general solution with poor performance and a specific solution with good performance. You can have all the specific solutions you need in one place and automatically dispatch to the right one without any extra effort.

I know it sounds too good to be true, but my limited experience so far is that it really is true, with surprisingly few pitfalls (mostly, you have to think about type instability, which will be new to most, but is really not that hard to avoid).

cbkeller··on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R
Doesn't that validate the applicability of the benchmark then? GP said:

> A typical user will rarely ever use the built in CSV library in Python

as a criticism of the benchmark, but if a typical user does use stdlib’s csv after all, then it seems like your disagreement is perhaps with GP and not with Stefan.

cbkeller··on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R
I mean, A lot of people did! HPC, DiffEq, etc.* And to some degree it's these communities who are now adopting Julia, interestingly enough.

*Elaborated in my other comment on GP https://news.ycombinator.com/item?id=24748107

cbkeller··on Nilang.jl – A Reversible Julia DSL
As far as I understand it (which is not very far):

Zygote calculates derivatives using source-to-source automatic differentiation.

This calculates function inverses (so to stretch the analogy a bit, it's kinda like "source-to-source automatic inversion")

cbkeller··on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R
I see what you mean, but from my perspective what this argument is missing is the context that a large and important subset of the scientific community has never moved to Python to begin with.

In other words, there is an even crankier and older community for which Julia may be the first actual change in a very long time.

This is particularly the case in HPC, where everyone still uses Fortran, C, or C++, and has never moved nor ever will move to Python because Python is fundamentally unsuited to these workloads. But in some cases, Julia is [1].

The best differential equation solvers (e.g., SUNDIALS [2] for a modern example) have been written in FORTRAN for the last 50 years. If Julia can challenge Fortran as the go-to language for this type of work (e.g. DifferentialEquations.jl [3] and SciML), that would hardly count as excessive churn.

[1] https://github.com/jeff-regier/Celeste.jl

[2] https://computing.llnl.gov/projects/sundials

[3] https://github.com/SciML/DifferentialEquations.jl

cbkeller··on Natural Nuclear Fission Reactor
Geologist here! Earth's internal heat budget [1] is a really cool subject since it is surprisingly hard to constrain, yet influences the operation of pretty much the entire solid Earth system (plate tectonics is entirely driven by secular cooling and radiogenic heat production).

Even using the current best estimate that about half of the current 42-46 TW geothermal heat flux is primordial and half is radiogenic, there's a bit of a mismatch if you try to extrapolate back in time* the solution to which may involve either a hotter core than previously expected, anomalously high present heat flux, or more errors in the way we're currently modelling heat loss through the mantle.

*To put it simply, imagine stepping back 1 Myr from the present. The ~half of the heat lost over that period that was _not_ radiogenic came from cooling the core and mantle, so the core and mantle should have been hotter 1 Myr ago. But if the mantle was hotter, it should have been less viscous -> convecting faster -> losing heat even faster in the past. If you keep extrapolating like this, it doesn't take too long for the extrapolated mantle to become physically implausible -- only about 2 or 3 Gyr out of the 4.56 Gyr we have to work with. There are various ways to solve this problem, but we don't know which solution or combination of solutions is what really happened.

Unfortunately there isn't a really good recent review paper on this, but for anyone interested here's a scattering of citations [2-5] that have most of the components to piece together.

[1] https://en.wikipedia.org/wiki/Earth's_internal_heat_budget

[2] doi.org/10.1029/JB085iB05p02517

[3] doi.org/10.1029/2003GL016982

[4] doi.org/10.1016/j.pepi.2020.106457

[5] scholar.google.com/scholar?cluster=7130495919742279897&hl=en

cbkeller··on The unreasonable effectiveness of the Julia programming language
Yeah, I think type instability is definitely the biggest culprit when someone's ported over some Python or Matlab code for the first time and isn't getting as much of a speedup as expected. It isn't hard to fix, you just have to know about it (and how to check for it with `@code_warntype`).

That and perhaps the excess unnecessary allocations from array indexing on the right hand side of an assignment, if you don't know about `view`s.

cbkeller··on The unreasonable effectiveness of the Julia programming language
I had an experience slightly like that at first where I was surprised to only be going 1-2x faster than my old MATLAB, but then I realized my code was full of trivially avoidable type instabilities and got another 100x speedup in Julia.
cbkeller··on A quick introduction to data parallelism in Julia
Tkf has so many cool packages! Less to do with transducers or parallelism, but in terms of just cool and useful stuff, Maybe [1] and BangBang [2] definitely come to mind as well.

[1] https://github.com/tkf/Maybe.jl

[2] https://github.com/JuliaFolds/BangBang.jl

cbkeller··on Update on Firefox Send and Firefox Notes
Thank you for building this and keeping it running!
cbkeller··on Trump gives Microsoft 45 days to clinch TikTok deal
"Favor" is the wrong word, but only because that implies a lack of payment. Quoting from https://news.ycombinator.com/item?id=18411779 :

> 2009 - NSA offering 'billions' for Skype eavesdrop solution [1]

> 2011 - Microsoft buys Skype for $8.5 billion. Why, exactly? [2]

> 2012 - Skype replaces P2P supernodes with Linux boxes hosted by Microsoft [3]

> 2013 - Microsoft handed the NSA access to encrypted messages Subhead: Skype worked to enable Prism collection of video calls [4]

> 1. https://www.theregister.com/2009/02/12/nsa_offers_billions_f...

> 2. https://www.wired.com/2011/05/microsoft-buys-skype-2/

> 3. https://arstechnica.com/information-technology/2012/05/skype...

> 4. https://www.theguardian.com/world/2013/jul/11/microsoft-nsa-...

cbkeller··on Multiple Dispatch in Julia
It exists! `@which fn(a,b,c)` will do exactly that. There's also `@less fn(a,b,c)` if you want to take quick look at the source code for the relevant method, or `@edit fn(a,b,c)` to open the relevant source file in an editor.
cbkeller··on Julia as a CLI Calculator
Yeah, absolutely
cbkeller··on Japan Captures TOP500 Crown with Arm-Powered Supercomputer
Looking into Amazon's power bill might be a useful start: Fugaku is listed as drawing 28 MW in OP. It's more power efficient than most, but to an order of magnitude that's a number we can work with. Amazon's power usage for US-East was estimated at 1.06 GW in 2017 [1] (at which time they also apparently owned about a gigawatt of renewable generating capacity [2], now closer to 2 GW [3]).

Either way you slice it, Amazon likely owns at least an order of magnitude more FLOPS than any single system on the top500. What they presumably don't have is the low latency interconnects, etc., needed for traditional supercomputing.

[1] https://datacenterfrontier.com/amazon-approaches-1-gigawatt-...

[2] https://www.eenews.net/stories/1060048034

[3] https://sustainability.aboutamazon.com/sustainable-operation...

cbkeller··on Japan Captures TOP500 Crown with Arm-Powered Supercomputer
Since Fugaku has been in the works for a while, I wonder if Apple just tried to choose the date of their announcement to coincide with the Top500 ranking
cbkeller··on Grassmann.jl A\b 3x faster than Julia's StaticArrays.jl
People often talk about speed as an advantage of Julia (which is true), but the real secret advantage compared to any other language I've worked with is more of a network effect: composability

I didn't appreciate the importance of this (or the degree to which this just works as a result of multiple dispatch), but there's a good talk about this from JuliaCon 2019 [1] by Stefan Karpinski

[1] https://www.youtube.com/watch?v=kc9HwsxE1OY

cbkeller··on IBM Releases Fully Homomorphic Encryption Toolkit for macOS and iOS
Was just about to post this here if no one else had yet. Really cool stuff! This was the first example I saw of FHE where I was actually able to easily run the underlying demo code [1] on my own system.

[1] https://github.com/JuliaCrypto/ToyFHE.jl

cbkeller··on Perchlorate, used in rocket fuels, may be more hazardous than previously thought
Both contain Cl but perchlorate is a highly oxidizing anion (ClO4-), while perchloroethylene is relatively unreactive (though carcinogenic) organic compound (Cl2C=CCl2)
cbkeller··on Julialang Antipatterns
Debugging slurm/pbs/cluster issues is no joke, but there are a few folks on Slack with relevant experience
← PreviousPage 7 of 14Next →