HNHacker News
TopNewBestAskShowJobs

jofer

4,312 karma · joined July 2, 2010

I'm a geologist (or a geophysist, take your pick). These days I get to work on some really neat remote sensing and image processing problems at Planet. Opinions very much my own, however. SLC-based, formerly Houston, TX.

meet.hn/city/40.7596198,-111.886797/Salt-Lake-City

Socials:

- bsky.app/profile/joferkington

- github.com/joferkington

---

submissionscomments
jofer··on The Art of the Brew: Exploring Hops and Other Plant Ingredients That Define Beer
I agree, but to be fair, one of the largest roles water chemistry plays is accentuating hop bitterness or dialing it back. Gypsum (CaSO4) will accentuate hops. Calcium Chloride will not as much, but changes mouthfeel and emphasizes malts more. At a basic level, you're changing the ratio of those two to emphasize hop vs malt character. So when discussing hops vs malts, I think it's fine to leave out water chemistry, because it's largely about doing the same thing - emphasizing different aspects of hop vs malt.

Other than that, it's largely a matter of balancing out Ph. Dark roasted malts are more acidic, so you add things to make the water harder (i.e. higher Ph) via relatively neutral-tasting things like CaCO3. Light roasted malts need softer water to stay in the same ideal Ph range (slightly acidic) during the mashing process, and if you go too far out of that range you'll get less than ideal results even with the same mash temps. That's not to say there aren't other aspects, but SO4 vs Cl ratio and overall water hardness for Ph control are the two main "knobs" in water chemistry for brewing.

The more interesting thing is the role water chemistry plays on the regional history of beers. In the past, you couldn't easily alter water chemistry. As a result, you see darker beers dominate where you have limestone aquifers with harder water that's ideal to offset the acid in dark roasted malts (e.g. lots of the UK) and lighter beers dominate where you have softer water in sandstone aquifers that's better for brewing them (e.g. lots of Germany and Czechia). It's hard to make a nice clean pilsner with hard water from a limestone aquifer, and you're not going to make a good porter with soft water from a sandstone aquifer.

So as a geologist, I've always found it kind of neat that geology is a core part of why certain beer styles arise in certain areas.

jofer··on Satellites Spotting Aircraft
"Orthorectified" (i.e. on a map) means both. It's corrected to a perpendicular to the observer view _and_ also corrected for scale distortion due to elevation (the top of a mountain is closer to the sensor and therefore appears larger than the valley beside it). For off-nadir imagery (i.e. not looking straight down) the view angle plays a larger role than the scale distortion part.

With that said, raw elevation matters too, especially when the imagery is off-nadir.

RPCs describe a view direction + lens distortion / etc. A completely flat surface at elevation will be in a different location than a completely flat surface at sea level. Things will also be a different _size_ as well depending on elevation.

Think of the RPC as a vector looking a specific direction (at a single pixel level, that's kind of what it is, though each pixel gets a different one). You're placing a point on the map where that ray intersects the Earth's surface. It intersects at a different point depending on the elevation of the surface. Projected (aka orthorectified) data needs to correct all of that to be able to place imagery on a map accurately.

If the airport is completely flat, then using a constant elevation is fine, but you still need to use that elevation and not assume it's at sea level. Otherwise it will be in the wrong place on a map and also a slightly incorrect size.

jofer··on Satellites Spotting Aircraft
That's a point that far too few people in the remote sensing industry understand!!

Put another way, try measuring the width of small objects in the scene. You'll find there's a minimum width things will appear regardless of how small they actually are. Small, high-contrast objects will be visible, but will be wider than they actually are. Measuring the width of small bright objects is one way of estimating the spatial resolution (i.e. FWHM of the PSF) of optical imagery.

And with that said, those fenceposts are not that thin. They're likely on the order of 10cm. It's widely assumed from regulations/etc that "spy" sats can get below 10cm spatial resolution. In the US, commercial sats are not allowed to collect imagery better than 10cm spatial resolution. At the 10cm resolution point, things like atmospheric lensing due to temperature variations become major issues and need to be corrected for (e.g. that "shimmer" you see above the pavement on a hot day). That type of tech gets tightly regulated very quickly even in the US's current "let private industry image how they want" regulatory environment (used to be more restrictive not long ago).

So that imagery is likely somewhere on the order of 5cm to 10cm resolution in its native form. Which is pretty nuts. It's crazy what the NGA can do!

jofer··on Satellites Spotting Aircraft
Quick geospatial note that's important for accurately geolocating these images:

You need a DEM to use RPCs for geolocation. Running things through gdalwarp as was done here will assume no terrain and 0 elevation. That will lead to significant mislocations anytime it's not flat and at sea level, especially given the off-nadir view angle of the data used here.

In other words:

  gdalwarp \
     -t_srs "EPSG:4326" \
     2024-05-25-15-37-54_UMBRA-06_GEC.tif \
     warped.tif
Should be:

  gdalwarp \
     -to RPC_DEM=some_dem.tif \
     -rpc \
     -t_srs "EPSG:4326" \
     2024-05-25-15-37-54_UMBRA-06_GEC.tif \
     warped.tif
If you don't want to use a DEM for orthographic corrections, then you should at least include a constant elevation in meters of the scene with RPC_HEIGHT. Otherwise things can be shifted kilometers from where the image actually is.
jofer··on US hospital told family their daughter had checked out when in fact she'd died
You'd be amazed how bad hospitals can be at keeping track of fairly critical things.

I had a cousin who voluntarily checked himself into the hospital related to severe withdrawal symptoms (and had bouts of psychosis related to said with withdrawal - meth addictions are nasty). His mother stayed for awhile but had to leave for work. She was repeatedly guaranteed by multiple doctors and nurses that he could not check himself out and would be there for several days at least. They told her to come back the next morning.

Late that night, he was discharged. Not even "he checked himself out". The hospital discharged him (and that's over an hour ride away from home, btw - no hospitals in rural areas these days).

He had no phone and no wallet. According to staff, he tried to call his mother to pick him up, but couldn't remember her cell phone number (it had changed recently). He called his sister and left a message, but she was traveling and didn't get it until much later. According to the hospital, he tried to call several different numbers trying to get a ride home and they made him stop and made him leave. In a quite rough downtown area he was not familiar with. While clearly not in his right mind. At 2am. With no wallet, no phone, no nothing.

The hospital had his mother's contact info. They did not give it to him even when he requested it. They did not attempt to contact her in any way.

We still don't know what happened afterwards. His body was found four days later in the river and it had been there for awhile.

Forcibly discharging someone under those circumstances and refusing to even contact their emergency contacts is beyond belief. I'm furious about it. Apparently it's common and not even something there's any recourse for.

jofer··on National Park Service Will Cite AWD Drivers for Driving on 4WD-Only Trails
Yeah, I came across that earlier. Definitely impressive!!

I completely agree with your general point (articulation matters a ton!) but I have to take a bit of issue with:

> "The details of your AWD or 4WD system don't matter as much if you can keep your wheels on the ground."

That's where the details of the system matter the most. Getting torque to a single tire is the hard part and the reason folks focus on it so much. A _very good_ AWD system can get enough torque to move the vehicle uphill / out of a tough system to a _single wheel_. Most can't. Most traditional part-time 4WD systems can't either. Open front and rear diffs are the norm in "true 4wd". Locking rear diffs are starting to become commonplace, but only a few stock vehicles come "triple locked" from the factory.

I grew up wheeling an old mid 80's S10 Blazer. Fun, small, fit down trails well, had the "holy grail" 100" wheelbase. Solid rear axle + IFS. Manual transmission. Good (enough) articulation. Plenty of clearance. Big enough tires. Crap horsepower. Worse gas milage. "True" part time 4wd with a transfer case (i.e. would 100% meet the NPS's definition in this case). But open front and rear diffs. I got stuck every time I got a tire off the ground.

I've taken the little bronco sport plenty of places I tried but could never make it in the Blazer. (And to be fair, vice versa... Big muddy ruts are not something I want to put the sport through for clearance reasons, and that's kinda what the Blazer did best.) At any rate, good modern AWD can often beat traditional part time 4wd with open front/rear diffs when wheel lift comes into play. On the flip side, independent suspension all around means it's going to lift tires _all the time_, so it _has_ to be good at it. Most AWD systems unfortunately aren't, even though some are these days.

jofer··on National Park Service Will Cite AWD Drivers for Driving on 4WD-Only Trails
Not by that definition, no.
jofer··on National Park Service Will Cite AWD Drivers for Driving on 4WD-Only Trails
https://www.nps.gov/cany/learn/management/compendium.htm

> High Clearance Four-Wheel-Drive (4WD) Vehicles

> A Jeep, sport utility vehicle (SUV), or truck type with at least 15-inch tire rims and at least eight inches of clearance from the lowest point of the frame, body, suspension, or differential to the ground. Four wheel drive vehicles have a driveshaft that can directly power each wheel at the same time and a transfer case that can shift between powering two wheel or four wheels in low or high gear. All wheel drive (AWD) vehicles do not meet this definition.

I completely agree they'll use their discretion, but either way, that definition is specifically a part time 4WD system with low range.

jofer··on National Park Service Will Cite AWD Drivers for Driving on 4WD-Only Trails
NPS defines 4WD specifically as part time four wheel drive with a transfer case and low range, FWIW. Low range is a big part of it. And yes, their definition technically excludes higher end "full time 4WD" systems in some cases, though I suspect everyone would look the other way at those.
jofer··on National Park Service Will Cite AWD Drivers for Driving on 4WD-Only Trails
AWD has come a long way in that regard in the last few years. It's still highly variable from manufacturer to manufacturer, but systems that use internal clutches alongside brakes (and not only brakes) to control wheel movement + tight feedback loops can really do a great job of minimizing wheel spin.

They get a lot of hate, but the bronco sport has the best AWD system I've driven to date in that regard.

And with that said, it is still the type of thing the Park Service would rightfully cite as not a proper 4wd. 9ish inches of clearance is not much, and the lack of a low range will bite you. I've taken mine on plenty of milder 4wd only trails in parks (e.g. black gap in big bend plus tons and tons of forest service roads), but I'm certainly not going to do elephant hill in canyonlands with it. That's what the dedicated off-road rig is for.

There are "4wd only" trails in national parks that high clearance AWD is fine on. The rangers will tell you which ones those are.

Canyonlands is a different beast than most national parks. Canyonlands has some very gnarly trails open if you have a permit. Lookup dollhouse sometime. Beautiful, but insanely technical. Elephant hill is better known and a bit milder.

jofer··on National Park Service Will Cite AWD Drivers for Driving on 4WD-Only Trails
Most stock pickups, actually. You'd be surprised how low the clearance is on a stock f150 without the various off-road packages. I.e. you can easily have a pickup with basic 4wd but only 8 inches of ground clearance. That's technically "high clearance", but not by much, and the poor approach, departure, and break over angles make it tough too.
jofer··on Show HN: I built an open-source tool to make on-call suck less
Yeah, I completely agree. I just meant that alerts you don't immediately respond to or are "noisy" aren't necessarily things you want to delete. Having low priority "noisy" alerts is not a bad thing.
jofer··on Show HN: I built an open-source tool to make on-call suck less
There is a lot to be said for "smoke test" metrics. Things you expect to have frequent false positives, but are sometimes early indicators of larger problems or indicators of where to look deeper if something else goes sideways. They're not things that should wake you up in the middle of the night, but they're a damn valuable tool to quickly figure out what's actually wrong when a "real" alert triggers.

Many of these lend themselves well to dashboards instead of alerts, but not everything is "dashboardable". Sometimes it's good to have a set of low-priority alerts that are treated differently than others.

E.g. "we're not receiving any data/requests". Sometimes that's just a lull in activity. Maybe a holiday. Sometimes it's because everything _else_ is broken and nothing is getting in (e.g. DNS issues).

With that said, I do think that classification should be made manually and not automatically.

jofer··on 370-mile hydrogen-electric seaplane set to clean up island hopping
That's largely because it's a byproduct of common hydrocarbon refining techniques. It's a bit silly to throw that away once it's produced.
jofer··on 370-mile hydrogen-electric seaplane set to clean up island hopping
There are lots of different types of hydrogen production, but many of them are "clean" (or at least as clean as solar, anyway).

The main issue with hydrogen is storage and transport. It's not an unsolvable problem, by any means, but it's significantly more difficult to safely store and transport than liquid hydrocarbons, at any rate. Hydrogen requires significant investments in infrastructure around it which mostly haven't been made yet.

jofer··on Multisatellite data depicts a record-breaking methane leak from a well blowout
It's a blowout. That means what it sounds like. An explosion and a fire. To fix it, you have to put out the fire. Once you put out the fire, you're still releasing methane.

Methane is flared instead of released under normal conditions if it's not being captured. A blowout is very much not normal conditions.

jofer··on What would it take to recreate Bell Labs?
Big oil arguably had some of the closest up until the past ~10 years.

I think folks may not realize how large Exxon and Shell's pure R&D labs are/were. The other majors have cut much more heavily on pure R&D in the past 20 years, but Exxon and Shell have always focused on pure research fairly heavily. It's been cut back in the past 10-ish years, but the R&D labs at both are still present. The breadth of it is significant, and while sure, it's geology, metallurgy, chemistry, etc (i.e. it's not what folks here likely think of as "tech"), it is fundamental research as opposed to applied research. A lot doesn't sure up under their R&D best because it's combined with internal consulting, but it's still a large group of folks hired to do r&d with diverse backgrounds.

Those labs still exist today, but they're not as large as what they were in the recent past (e.g. 2010), and especially compared to their heyday in the 70's.

jofer··on Introduction to Bash Scripting
Shell pops up in a ton of places even where it's not used directly. Build systems are a good example. I have yet to come across something that doesn't touch shell at some level unless it also doesn't touch an OS. But maybe that's just me. Phones aren't an exception to that, though.
jofer··on Introduction to Bash Scripting
Can you seriously name something other than very low level embedded software that doesn't involve shell at some level? I certainly can't...
jofer··on Introduction to Bash Scripting
Any time you're running lots of CLI utilities directly is a good start. Bash is very very good for that, and other than perl, most other things aren't particularly good at it.
jofer··on Introduction to Bash Scripting
For anyone wondering whether or not you really need to know bash: Yes. Yes, you do.

Somewhere in the middle of every stack is a bunch of bash and makefiles. Don't replace that or avoid it. Embrace it. It's not perfect, but it's important to know.

jofer··on Notebooks Are McDonalds of Code
You're not editing code in the CLI. You're editing code in your preferred editor/IDE. ipython + %edit works wonders. That's one of the main reasons to avoid notebooks and stick with CLI-based or similar tools.

Jumping into the middle of a script and editing is often a bit dangerous. You usually wind up with something that can't be reproduced, as it depends on modifying state in a way that wasn't recorded. You do often need some sort of caching during exploration, for sure (it's not really feasible to wait an hour every time you want to tweak a visualization), but it's often better to break pipelines up a bit and do disk I/O rather than leaning on cells in a notebook.

I do agree on the remote client part, though. Browser-based interactivity often is a good solution for a dev instance + a local thin client. It's slower than native for a local solution, though.

jofer··on Notebooks Are McDonalds of Code
So have I (not FAANG, though). I've been working in scientific python for over 20 years and long before notebooks came to the python world. (Yes, pre-numpy) And I've seen notebooks abused far more than used well. The same general thing was heavily abused in Matlab/etc too. It's not only Jupyter and it's not new.

It's not that they're not a good tool, it's that they're the wrong tool for a lot of what they're used for. They're an excellent teaching and documentation tool. However, they're remarkably bad for actually doing any type of analysis or exploration. They're a great way of presenting your analysis after you complete it, but they're bad for actually doing that analysis. Write analysis code and data exploration code using the same tools you would for production code. There's nothing wrong with an edit + run + explore loop. Do that with a standard REPL like ipython + your preferred editor.

But there's a lot wrong with trying to use a notebook to write and execute code, as it: 1) encourages writing code that won't actually run later (dependent on non-linear code execution and unsaved changes made to earlier cells), and 2) is a poor environment for writing code compared to more full featured editors or IDEs, and 3) discourages interactivity and data exploration due to being browser-based (poor performance compared to running directly limits interactivity during data exploration).

Your data analysis work needs to actually run independently later. Notebooks rarely do unless they're re-made from scratch. At that point, you're better off working in a more full-featured environment for the exploration phase and then using the notebook as later documentation / communication.

jofer··on Notebooks Are McDonalds of Code
Notebooks are primarily a _teaching_ tool, not code or an experimentation interface. I know that's not how they're used, but it's really bizarre to me that people use them for exploratory data analysis. They're great for teaching and for _documenting_ the results of exploration, but fairly bad for any type of interactive data exploration, let alone actually writing code.

Frankly, notebooks are pretty terrible for exploratory data analysis, especially in python. The main issue is that they absolutely kill interactivity. Straight up CLI based ipython is far more interactive, and often better for exploratory data analysis. In a lot of ways, pretty much any other interface is better than code blocks that aren't independent but can be run independently.

Part of it is the interface (i.e. non-linear state, code blocks, etc), which is great for teaching, but bad for exploration. However, i.m.o. even more of it is the fact that it's browser based. E.g. folks often think it's best to use matplotlib in a notebook. However, using a notebook means you get very little interactivity out of a highly interactive library and remarkably poor performance out of something that's actually _very_ interactive and quite performant on standard Tk/Qt/etc backends.

If you ditch browser based stuff, you're usually far better off. This part is mostly just me griping, but I don't understand why we've stepped back 20 years in basic usability, interactivity, and performance by making everything use a web browser (and then making that not even remotely cross platform).

jofer··on Water is bursting from another abandoned West Texas oil well, continuing a trend
Because volume expansion from a reaction wouldn't be that large. Things aren't burning/etc.

Direct injection of large volumes of water (with those things in it) is more than enough to explain it. No need for reactions of any sort.

The issue is that the injection should be into far deeper sections. Something in the active injectors is leaking and therefore the injected liquids aren't making it to the bottom of the well and are flowing out the sides and into shallower permeable sections. The simplest explanation would be a bad casing (i.e. the steel sides of the well) in a major disposal well, but it's likely more convoluted than that. Either way, it's a significant problem for a lot of reasons.

jofer··on A 100x speedup with unsafe Python
Ah, right, you mean _outside_ the memory block of the array! Sorry, my mind was just foggy this morning.

That's not strictly possible, but "circular" references are with "stride tricks". Those can accomplish similar things in some circumstances. But with that said, I don't think that would work in this case.

jofer··on A 100x speedup with unsafe Python
You actually can do the "offset by 2 bytes back" with a reshape + indexing + reshape back. But I suspect I'm still missing something. I need to read things in more depth and try it out locally.

By "numbs" here, I'm assuming you mean numba. If that's not correct, I apologize in advance!

There are several things where Cython is a better fit than numba (and to be fair, several cases where the opposite is true). There are two that stick out in my mind:

First off, the optimizations for cython are explicit and a matter of how you implement the actual code. It's a separate language (technically a superset of python). There's no "magic". That's often a significant advantage in and of itself. Personally, I often find larger speedups with Cython, but then again, I'm more familiar with it, and understand what settings to turn on/off in different situations. Numba is much more of a black box given than it's a JIT compiler. With that said, it can also do things that Cython can't _because_ it's a JIT compiler.

If you want to run python code as-is, then yeah, numba is the better choice. You won't see a speedup at all with Cython unless you change the way you've written things.

The second key thing is one that's likely the most overlooked. Cython is arguably the best way to write a python wrapper around C code where you want to expose things as numpy arrays. It takes a _huge_ amount of the boilerplate out. That alone makes it worth learning, i.m.o.

jofer··on A 100x speedup with unsafe Python
I'm confused on the need for cytpes/etc here. You can directly modify somearr.strides and somearr.shape. And if you need to set them both together, then there's numpy.lib.stride_tricks.as_strided. Unless I'm missing something, you can do the same assignment that ctypes is used for here directly in numpy. But I might also be missing a lot - I'm still pre-coffee.

On a different note, I'm surprised no one has mentioned Cython in this context. This is distinct from, but broadly related to things like @cython.boundscheck(False). Cython is _really_ handy, and it's a shame that it's kinda fallen out of favor in some ways lately.

jofer··on Earth Online: non-stop satellite monitoring platform
It's also based on open and freely accessible data: Sentinel2.

That's not to take anything away from what they're doing. They've done a very very nice job. But there's a precedent for folks expecting open data and paying for additional services/etc.

jofer··on What is Webb observing now?
I'm not an astronomer, but I believe the "mottled red" features are distant objects that are redshifted. Note that they're almost all red no matter where you zoom in. That's the evidence that space is expanding in a nutshell.
← PreviousPage 4 of 22Next →