5,360 karma · joined March 9, 2008
@kmx
Previously @twittereng, websolr.com, bonsai.io, Houseparty, etc
kyle@kylemaxwell.com
Who knew that densely planted tall pine trees burn so much?
There are in my mind two reasons to not use PG's search.
1. Elasticsearch allows you to build sophisticated linguistic and feature scoring pipelines to optimize your search quality. This is not a typical use case in PG.
2. Your primary database is usually your scaling bottleneck even without adding a relatively expensive search workload into the mix. A full-text search tends to be around as expensive as a 5% table scan of the related table. Most DBAs don't like large scan workloads.
The benefit of using S3 in that case is that unlike e.g. Elastic, your block cache servers don't need replication, and you can tear them down when you're done. You can put them in a true autoscaling group as well.
What other typical company functions have this sort of provider?
Conversely, what other company functions need this sort of provider?
The goal of GQL is to gather a useful subset of your microservice data into a single tree for your frontend to render. In a facebook context, a typical query is "give me all the info i need to be able to show my home feed".
The goal of graph dbs is to allow arbitrary, complexm, often cyclic and aggregating queries to be run against a data source expressed as a graph. A typical query is something like "how many of my friend-of-friends like basketball, displayed by location."
Without the ability & incentive to hire someone "junior," every interviewer will compare the new hire to their inflated view of themselves.
Um, we're doing it wrong.
TinyGo is far better, but only supports a subset of Go's stdlib.
I'm playing with TinyGo, and using GPRC communication with the outer engine as a poor substitute for WASM interface types.
https://github.com/fizx/goingo if you want to read the ugly prototype!
I much prefer 5hrs/day of focus, arriving at the office refreshed, excited, and creative.
There's not a great reason for there to be "shards" and "clusters". In 2021, I'd appreciate search running from object storage (e.g. S3), with a dynamic compute layer over the top providing all of the compute--especially with indexing being such a source of variable compute.
Also, why do we have to re-index all of the time? Most textual customization can happen at query time, with only a mild performance hit.
I'm curious if we ever truly get a cloud-native, easy-to-maintain search thing, or if we have to wait for ML to eat search in order to get nextgen tech.
While they both netted positive outcomes for me, if I had gone all-in on either one, I think I would have had an order-of-magnitude better outcome. Splitting my attention cost me serious advancement & money at both.
https://blog.twitter.com/engineering/en_us/a/2010/introducin...
As you mention, the probability density converges to multiple outcomes, but then I believe that you're in more of an Everett Many Worlds between the outcomes, rather than a "choice."
In practice, it seems like all of the Many Worlds scenarios statistically converge into one world sooner or later anyways (would love this to be formally shown, but have never seen it, so it's just conjecture from me). For example, if you flip a quantum coin, splitting the universe in two in a Many Worlds Interpretation, who cares? Does the Sun notice? Does someone in a town a mile away?
You've created a small bubble of "two universes," in a Many Worlds sense, but that bubble will pop. Quantum mechanics is ambivalent about the direction of time, so it sure seems like worlds are joining as fast as they're splitting, keeping us statistically around one effective universe.
Ideally, a platform team should give you reliable, self-service components to build upon, like databases, caches, rate-limiters, api-gateways, etc.
If they're framework-ish, they're mandating you build your app in exactly their preferred way in order to receive the benefits.