juliansimioni

For me it was “the results speak for themselves” and then simply a (large) number of automated tests run that never had human eyes.

Yes, quantity famously has a quality all its own, but perhaps not where correctness checks for something this central is concerned.

I think the warning in the article is a little more subtle.

The author isn’t placing a moral judgement on certain goals or whether the result of reaching that goal has a large enough audience.

They are warning against doing things that work against achieving our goals in ways that are hard for us to notice by ourselves.

I got both (because of course I did) with no problem. The welcome Suica came with a free notebook which I’m also using to collect the Japan Rail stamps

https://www.jrailpass.com/blog/eki-train-stamps

It’s really a great conference, the OSM community is great and strongest in Europe. I’ve been to many and always come away inspired.

Sadly (or, really, amazingly as it’s good to have so many options), another great, major geospatial conference is also happening essentially at the same time: FOSS4G in Hiroshima. They are certainly different conferences but a lot of overlap.

So, in other words it’s a great week to be in geospatial software working on open source and open data.

I am literally on a flight to Japan right now, and one of my main goals is to add a Suica card to my collection of transit cards from around the world (currently 31 cards large).

Sadly physical cards may only have a few years left, for the most part using tap-to-ride is superior. So I really have to get out there and grab them while I can.

Yes. And the fab companies and their suppliers are deliberately and wisely slow to scale up production to meet short term changes in demand. They've seen the history of the semiconductor industry, it's constant boom and bust cycles. But they have the highest op-ex costs of anyone. So when the party's over they are the ones who pay for it the most.

Fellow bootstrapped founder for 8 years here, and I love and agree with your responses here.

Having previously existed in the VC-backed startup world, one thing I don't miss is the belief that its the _only_ rational way to run a business. In reality there are a lot of dangers to that approach, like you pointed out.

VCs _need_ promising businesses to join their portfolio, so they'll always be trying to convince you to raise money and have a tiny shot at making it big. If you fail, well, you're just one business in their portfolio, another one will pick up the loss. But it's the _only_ company you have, so you are doing the right thing by growing sustainably.

There's something extremely freeing about running a bootstrapped business and knowing you don't _need_ anyone to keep it running. Cheers to the next 10 years for you and your team.

I think it's more than unwilling, we're straight up unable. Not in principle, but because it's a coordination problem.

Think of how many parents now want their kids not to have a smartphone or social media. There's genuine, well researched evidence that this would be good. But there's also real harm to kids who see all their friends with iPhones and Instagram. It might sound silly to us but it's definitely real feeling to the kids.

A lot of the parents who let their kids use smartphones and social media probably could be easily convinced that it's a bad idea, they just don't know. Or they don't know how bad it is, and so something else, like displaying status via a nice phone, they value more highly.

But until we can reach critical mass, and fight the (not insignificant, and quite intentional) momentum to use social media, it will be hard.

In my social and meetup circle, COVID coincided with all the organizers of my favorite meetups reaching the age where one generally starts to slow down, have kids, and possibly move out of the city for more space and calm.

At that point they had all been running their meetups for 5-10 years, so they were also established in their careers and didn't NEED them anymore. So while keeping an existing meetup running pre-COVID was not exactly easy, it was doable and familiar.

But now, 5 years after COVID, all the longstanding meetups are long enough gone that people nearly have to start from scratch. It's starting to happen, but is a huge uphill battle. For the most part, I suspect it will be a new generation of organizers, but they'll be learning all the same lessons again, in a much less interesting and more hostile tech environment.

Similar to this: if you want to optimize your productivity*, do so on a timescale of at least weeks if not months or years.

Simple example: Can you get more done working 12 hours a day than 8? Sure, for the first day. Second day maybe. But after weeks, you're worse off in one way or another.

It's easy to chase imaginary gains like automating repetitive tasks that don't actually materialize, but some basics like sleep, nutrition, happiness, etc are 100% going to affect you going forward.

* I actually hate that word, and prefer saying "effectiveness". Productivity implies the only objective is more, more, more, endlessly. Effectiveness opens up the possibility that you achieve better results with less.

This is super cool. As part of the Pelias geocoder(https://pelias.io/) we use both OSM and SQLite heavily. Currently we've written our own pbf2json tool in Golang (https://github.com/pelias/pbf2json). But creating intermediate databases in SQLite could enable more powerful manipulation of OSM data before we eventually import it.

It's wild, and absolutely worth writing about, that at some point in recent years, the concept of "AFK" basically ceased to exist.

Yes, we aren't technically near a keyboard most of the time today, but we are never AFK in a conceptual sense. Even when sleeping.

It's interesting that internet infrastructure is so interconnected that failures of what _should_ be separate systems seem to coincide.

Do we think it's possible the GCP outage caused the Cloudflare outage, or vice versa?

For DownDetector, I know it's not exactly foolproof. I imagine a lot of sites use something like CloudFlare in front of AWS infra, which could easily be automatically flagged as an outage for both in their system.

Another vote to confirm that AWS appears to be working fine. At least EC2 and S3 which is basically all we use (highly recommend this simple setup using base primitives, as some of their higher level services have significantly lower reliability)

>Renewable energy in Japan will receive a *seismic shift*

Maybe not the best analogy for the most earthquake-prone country in the world?

Like many, I also had an experience like this in my younger days: obviously unfit product, early prototype made by bottom-of-the-barrel contractors, co-founders who can't code, no salary, no users.

I got lucky, and spent only a few months while not working that hard.

But at the same time, this quote hits home:

>I was doing a startup. I was executing, and for the first time in my professional life I wasn’t insulated from the results. I didn’t achieve my destiny of great things, but I’d built something.

Geonames is a great dataset, in fact it's one of the "OG" open-source databases of the modern era, dating back to 2005.

It has fairly comprehensive coverage of countries, cities, and major landmarks. It also has stable, simple identifiers that are somewhat of a lingua-franca in the geospatial data world (i.e. Geonames ID 5139572 points to the Statue of Liberty and if you have other data that you need to unambiguously associate with the one Statue of Liberty in New York Harbor, putting a `geonames_id` column in your database with that integer will pretty much solve it, and will allow anyone else you work with to understand the connection clearly too).

However, to be honest, it hasn't really kept pace with modern times. The velocity of changes and updates is pretty low, it doesn't actively grow the community anymore. The data format is simple and rigid and built on old tech that's increasingly hard to work with. You can trust Geonames to have the Statue of Liberty, but not the latest restaurants in NYC.

For a problem like the post author has of finding ways everyday people can easily navigate to something like a park bench that might not have a single address associated with it, or even if it does, needs more granularity to find _that_ specific bench in a park with 100 benches, Geonames probably won't help.

Source: I'm co-founder of Geocode Earth, one of the geocoding companies linked in the blog post. We use Geonames as one source of POI data amongst many others.

AFAIK, a lot of the RAM requirements come from importing and processing the data. It already takes quite a long time and would be even slower without heavily utilizing RAM.

Nominatim also doesn't support any sort of typeahead queries. There's Photon(https://github.com/komoot/photon), which works in concert with Nominatim and is similarly tied to OSM as a data source.

There's also Pelias(https://pelias.io/), an open-source geocoder that handles all types of geocoding in one and supports OSM, OpenAddresses and many other sources (including custom data) out of the box. Admittedly it can have more moving parts as a result, but it can be set up with relatively little RAM if you're careful (I bet a planet install could be done somewhat slowly with 32GB RAM). Full disclosure, I have been a core-maintiner of the project since 2015.

Are you committed to running the process locally due to privacy or legal reasons?

If not, we securely run geocoding batches of that size at Geocode Earth all the time at pretty competitive rates. We are flexible on data transfer, usually we have customers set up an SFTP server or an S3 bucket and send us the credentials. We spin up a ton of hardware in EC2 to geocode it real fast (<24 hours even for a few hundred million addresses), will work with any data format you need, and then send it back.

If you _do_ need to run it locally, we're also the creators of the Pelias geocoder(https://pelias.io), which is open source like Nominatim but supports more than just OSM data (which is not a comprehensive address source globally), so often can return better results. We can help you set it up if you need.

Geocoding is a really fun (and sometimes frustrating) problem I've been lucky enough to have been working to solve for over 10 years now.

I joined Mapzen in 2015 which ostensibly was part of a Samsung startup accelerator, but looking back, it's more descriptive to say it was an open-source mapping software R&D lab. We built what is now foundational open-source geospatial tools like the Pelias geocoder (my team) and the Valhalla routing engine. A lot more projects like the Tangram map renderer are still really useful post-Mapzen.

A reasonable, but very wrong, first assumption about geocoding is that with a database of places you're almost there. Inputs are often structured, like some addresses, but the structure has so many edge cases you also have to effectively consider it unstructured. The data is the same, sometimes worse as a lot of data sources are quite bad.

Over the last 10 years we've explored most strategies for full text search, and no ONE solution knocks it out of the park. We started with really simple "bag of words" search, just looking at token matches. That, fairly predictably was mostly a mess. With billions of places in the world recorded in open datasets, there's going to be something irrelevant somewhere that matches, and probably drowns out whatever you're looking for.

Parsing inputs for structure is an enticing option too, but for any pattern you can come up with, there's either a search query or some data that will defeat that structure (try me).

The previous generation of ML and a lot of sweat by Al Barrentine produced libpostal(https://github.com/openvenues/libpostal), which is a really great full-text address parser. It's fast and accurate, but it doesn't handle partial inputs (like for autocomplete search), doesn't offer multiple parsing interpretations, and still isn't always right.

What we've settled on for now for autocomplete is a pretty sophisticated but manually configured parser, which can return multiple interpretations and is also quick to fall back to "i don't know" (how can you really parse meaning out of a short input like "123": is it the start of a postalcode? a housenumber? the name of a restaurant?). It's also runtime bound to make sure it always returns in a few milliseconds or less, since autocomplete is extremely latency sensitive. Then we can either search with the benefit of more structure, or worst case fall back to unstructured, with a LOT of custom logic, weights, filters, and other tricks as well.

A big question right now is will next generation LLMs completely solve geocoding, and honestly I'm not sure. Even older ML is really eager to over-generalize rules, and while newer LLMs do that less, they also still hallucinate, which is pretty much a dealbreaker for geocoding. At least for now LLMs are also orders of magnitude too slow, and would never be cost effective at current prices. Personally I think us geocodeurs will be in business a while longer.

There's so much more about geocoding I love talking about, it's truly a niche filled with niches all the way down. This is the sort of stuff we are always iterating on with our business Geocode Earth (https://geocode.earth/). We think we have a really compelling combination of functionality, quality, liberal usage license (hi simonw!), respect for privacy, and open-source commitment. We always love hearing from people interested in anything geocoding so say hello :)

"we need to talk about your on call volunteering"

"Really? I volunteered for 15 hours of on call"

"Well, 15 is the minimum ok? Now it's up to you if you just want to do the bare minimum, or uh..well look at Brian for example, he volunteered for 37 hours of on call"

What happens when you need to encode the newline character in your data? That makes splitting _either_ CSV or LDJSON files difficult.

There's also the infamous bucket brigade from Hurricane Sandy. There was a Stack Overflow podcast about it but it looks like it disappears, here's a read though:

https://www.datacenterdynamics.com/en/analysis/reliving-sand...

Yes, that was the interdictor blog!

https://interdictor.livejournal.com/2005/08/27/

High voltage direct current

So, to be clear, this is absolutely a case of these businesses making choices they know are harmful but are technically legal, and is bad.

But I wonder if in the long term adding trace amounts of common allergens to many foods is a viable strategy? I read recently that exposing kids to trace amounts of peanuts at a young age drastically cuts down on peanut allergies.

Maybe a better strategy is to make a pill with tiny amounts of common allergens (shellfish, peanut butter, sesame, celery, etc) and encourage parents to feed it to their kids daily?

I really want to see it explain the famous Quake fast inverse square root, but apparently it's extremely rate limited right now

Dogfooding can often be a good thing but not if your target customer is a very different type of business than your own.

For example, Stripe is a _very_ large company but their product is geared mostly towards small and medium size businesses.

It's very possible that in the process of making Stripe Billing work for Stripe, they'd make it work less well for their actual customers.

It's the same reason why Intuit probably doesn't use QuickBooks to do their accounting.

I just renewed my passport completely online through their beta program, and it was amazing. Only about two weeks from when I submitted the form to my new passport arriving. Hopefully that's the rough timeline for everyone going forward, but I wonder if they are able to process things faster right now due to lower volume.

Hi! Maintainer of Pelias here and co-founder of Geocode Earth where we continue to fund Pelias development through our SaaS and consulting.

You need to bring Who's on First data in to support administrative areas, it won't use admin areas from OSM. This is a bit of a technical choice (WOF has some nice features like stable IDs and a rich hierarchy, and a data format that's much easier to work with), but also the result of the Who's on First project and Pelias being developed together while we were at Mapzen.

Don't hesitate to reach out to us if you need help getting things set up.