2,385 karma · joined December 1, 2015
Postgres allows obtaining advisory locks at either the session _or_ transaction level. If it's session-level, then you have, ergo, a connection-level lock.
https://www.postgresql.org/docs/current/explicit-locking.htm...
- ok, a single ext4 file inode changes, and its filename matches my hardcoded string
- oh, you don’t want to match against just changes to “package.json” but you want to match against a regex? voila, now you need a regex engine
- what about handling a directory rename? should that trigger matches on all files in the renamed directory?
- should the file watcher be triggered once per file, or just every 5ms? turns out this depends on your use case
- how do symlinks fit into this story?
- let’s say i want to handle once every 5ms- how do i actually wait for 5ms? do i yield the thread? do i allow other async contexts to execute while i’m waiting? how do those contexts know when to execute and when to yield back to me? now you have an async runtime with timers
- how does buffering work? are there limits on how many file change events can be buffered? do i dynamically allocate more memory as more file changes get buffered? now you need a vector/arraylist implementation
And this is before you look at what this looks like on different platforms, or if you want polling fallbacks.
Can you do it with less dependencies? Probably, if you start making hard tradeoffs and adding even more complexity about what features you activate - but that only adds lines of code, it doesn’t remove them.
What you describe is ideologically nice, but in practice it’s over-optimizing for a goal that most people don’t really care about.
Here's a analogously real example from current US-Ukraine policy:
> For example, one current social goal in the U.S., given the geopolitical conflict with Russia, is to avoid facilitating activities that could aid the adversary. As Russia has invaded Ukraine, the U.S. has positioned itself in opposition to Russia but not Ukraine. Banks, therefore, need to align with these geopolitical stances, leading to decisions that might catch some individuals in the crossfire, even if they’re not directly involved.
> Financial institutions often interpret this as: if they're not deeply specialized in doing business in Ukraine, they should avoid it altogether. They fear they won’t be able to consistently ensure compliance with these complex directives from the government [especially because there's a chance those directives might change in a week, or a month, or 3 months].
> This creates a split-brain problem within U.S. decision-making. The government intends to say, "Please cut down on oligarch money laundering that supports Russia’s war effort." However, financial institutions hear this as, "Under no circumstances should you fund anything related to Ukraine," including, for example, scholarships for Ukrainian high schoolers—a slight exaggeration, but not far from the reality in some cases.
(source: https://www.complexsystemspodcast.com/episodes/true-crime-ba...)
It's vacuous, makes vague claims that don't leave room for proof/disproof, and doesn't offer any reason that it's any better than a prompt that asks GPT4o "was this generated by AI y/n"
There’s a lot that has to go into fixing things on such a tight timeline too:
- oncall-level alerting for your security.txt inbox - your oncall needs to either be someone who can actually take corrective action on the system in question (not easy in a large company!) or able to route the issue to the right team - the service owners need to be empowered to treat security with the appropriate severity (taking the site down so quickly speaks highly to this)
Hats off to the points.com team. With any luck, this post doesn’t get too much traction and y’all won’t get flooded with bounty beggar spam.
I’m now completely lost as to why you believe this was a good idea over using something like MySQL/Postgres/Aurora. As I see it, you’ve added complexity in three different dimensions (novel DB API, novel infra/maintenance, and novel oncall/incident response) with minimal gain in availability and no gain in performance. What am I missing?
(FWIW, I worked on Bigtable/Megastore/Spanner/Firestore in a previous job. I’m pretty familiar with what goes into consensus, although it’s been a few years since I’ve had to debug Paxos.)
> I was trying to drive home the point that you don't need a massively distributed system to make a useful startup. I think some founders go the opposite direction and try to build something that scales to a billion users before they even get their first user.
This reads to me as exactly the opposite: overengineering for a problem that you don’t have.
For exactly the reasons you describe, I would argue the burden of proof is on you to demonstrate why Redis, MySQL, Postgres, SQLite, and other comparable options are insufficient for your use case.
To offer you an example: let’s say your Big Customer decides “hey, let’s split our repo into N micro repos!” and they now want you to create N copies of their instance so they can split things up. As implemented, you’ll now need to implement a ton of custom logic for the necessary data transforms. With Postgres, there’s a really good chance you could do all of that by manipulating the backups with a few lines of SQL.
But I have some bad news: you haven’t built a system without a database, you’ve just built your own database without transactions and weak durability properties.
> Hold on, what if you’ve made changes since the last snapshot? And this is the clever bit: you ensure that every time you change parts of RAM, we write a transaction to disk.
This is actually not an easy thing to do. If your shutdowns are always clean SIGSTOPs, yes, you can reliably flush writes to disk. But if you get a SIGKILL at the wrong time, or don’t handle an io error correctly, you’re probably going to lose data. (Postgres’ 20-year fsync issue was one of these: https://archive.fosdem.org/2019/schedule/event/postgresql_fs...)
The open secret in database land is that for all we talk about transactional guarantees and durability, the reality is that those properties only start to show up in the very, very, _very_ long tail of edge cases, many of which are easily remedied by some combination of humans getting paged and end users developing workarounds (eg double entry bookkeeping). This is why MySQL’s default isolation level can lose writes: there are usually enough safeguards in any given system that it doesn’t matter.
A lot of what you’re describing as “database issues” problem don’t sound to me like DB issues, so much as latency issues caused by not colocating your service with your DB. By hand-rolling a DB implementation using Raft, you’ve also colocated storage with your service.
> Screenshotbot runs on their CI, so we get API requests 100s of times for every single commit and Pull Request.
I’m sorry, but I don’t think this was as persuasive as you meant it to be. This is the type of workload that, to be snarky about, I could run off my phone[0]
Apple also did that because it is a _business_, and has not always been the behemoth it is today. They went overseas because it made more sense, not because they had some anti-American goal of crippling US mfg capabilities.
The AELP is, ideologically, a anti-big-co institute, which is admirable but is also why this piece engages in post hoc ergo propter hoc rather than discussing meaningful policy proposals:
> The American Economic Liberties Project launched in February 2020 to help translate the intellectual victories of the anti-monopoly movement into momentum towards concrete, wide-ranging policy changes that begin to address today’s crisis of concentrated economic power
I haven't been in this situation before, but I imagine one of the trickier challenges in a role like what OP is describing is that whoever leads this also needs the ability to manage scope for the team, including persuading the existing COO or whatnot that "yes, we _could_ bring this in-house but you really really do not want to."
It did help that we wrote our compiler in Rust and were able to transpile it to WASM :)
constexpr does not mean that you can evaluate arbitrary C++ code at compile time. It allows you to evaluate a _very specific subset_ of C++ at compile time that is not at all easy to wrap your head around: look no further than https://en.cppreference.com/w/cpp/language/constexpr to understand the limitations.
It's also easy to read this as "interviewer hand-held a candidate through a problem".
On the other hand, the paper is kind of dismissive about engineering nuance and gets some details blatantly wrong.
- MapReduce is alive and well, it just has a different name now (for Googlers, that name is Flume). I'm pretty confident that your cloud bill - whether or not you use GCP, AWS, or Azure, is powered by a couple hundred, if not thousand, of jobs like this.
- Pretty sure anyone running in production has a hard serving dependency on Redis or Memcache _somewhere_ in their stack, because even if you're not using it directly, I would bet that one of your cloud service providers uses a distributed, shared-nothing KV cache under the hood.
- The vast majority of software is not backed by a truly serializable ACID database implementation.
-- MySQL's default isolation level has internal consistency violations[1] and its DDL is non-transactional.
-- The classic transaction example of a "bank transfer" is hilariously mis-representative - ACH is very obviously not implemented using an inter-bank database that supports serializable transactions.
-- A lot of search applications - I would venture to say most - don't need transactional semantics. Do you think Google Search is transactional? Or GitHub code search?
(Plus, more output tokens is more expensive!)
e.g.
using the cl_100k tokenizer (what GPT4 uses), this JSON is 60 tokens:
{
"method": "GET",
"endpoint": "/api/model/details",
"headers": {
"Authorization": "Bearer YOUR_ACCESS_TOKEN",
"Content-Type": "application/json"
},
"queryParams": {
"model_id": "12345"
}
}
<?xml version="1.0" encoding="UTF-8" ?>
<method>GET</method>
<endpoint>/api/model/details</endpoint>
<headers>
<Authorization>Bearer YOUR_ACCESS_TOKEN</Authorization>
<Content-Type>application/json</Content-Type>
</headers>
<queryParams>
<model_id>12345</model_id>
</queryParams>
That being said, if you have suggestions for how we can make this table more objective, we’re all ears!
If you’re driving user-facing interactions with LLMs, though, and you’re already dealing with >1min latency on the first call (as many of our current users are!), waiting for another LLM call to come back is a really frustrating thing to block your UX on.
LLMs today are really good at producing output that satisfies the very vague metric of “this looks good to a human” but aren’t nearly as good at producing output that satisfies a complex set of syntax and schema constraints. The state space of the former is much larger than the latter, so there’s a lot more opportunity for an LLM to be successful by targeting the state space of “looks good to a human”. Plus, there’s still a lot of room for advancement in multimodality and data quality improvements.
Search problems, in general, deal with this too: it’s easy to provide a good search experience when there are a lot of high-quality candidates, and much harder when there are fewer, because all you have to do is return just a few of the best candidates. (This is partly why Google Drive Search has always sucked compared to Web Search- it’s really hard to guess exactly which document in a 10k-file-Drive a user is looking for, as opposed to finding something on Wikipedia/NYTimes/Instagram that the user might be looking for!)
Out of curiosity- do those orgs not find the loss of generality that comes from custom models to be an issue? e.g. vs using Llama or Mistral or some other open model?
You can check it out over at https://github.com/BoundaryML/baml. Would love to talk if this is something that seems interesting!
(It’s almost the opposite, in fact, where our courts have expanded the Commerce Clause over time to allow the federal government the ability to regulate close to any economic activity.)
(Disclaimer: I’ve never had to exercise this personally.)