1,113 karma · joined July 29, 2016
Husband and Father. More @ https://rockwotj.com
They are yolo mode by default with periodic fsync and a big mutex around every reducer: https://strn.cat/posts/spacetime/ (granted things may have changed since that blog post)
> I can't recall seeing a network-bound cluster
I saw some of these (most packets per second not bandwidth) in the Firebase Realtime Database because changes get broadcast to many users. Since SpacetimeDB is made for games this is the same synchronization effect. Traditional databases don’t do this which is why Cockroach wouldn’t have seen it.
If I have the same number of CPU cores and they all can do their work in half the time they can double the number of requests now
Honestly the biggest limitation is its quite a big protocol and can be complicated, but I don’t see any major advantages you could do yourself protocolwise over a custom FUSE thing except for just simplification. In my case I am hoping the kernel client can save me a lot of work of building a good client
Can you elaborate on why? NFS is really a great protocol with a lot of tricks to reduce round trips
> Motivation for combining Chorus and SlateDB for NFS
I work on an AI agent (tasklet) and we give every agent a linux machine. Having durable storage that is cheap, fast and multi-tenant is really important for our product. NFS is a great protocol (if complicated), and object storage is just the cheapest. But making it fast and reliable is key.
> other use cases
Any use case for SlateDB that you are willing to pay more for less latency but keep disaggregated storage without another system.
> GCP specific
Actually AWS and Azure zonal storage also support append operations, so I think the approach could be extended to all three major clouds. I don’t have a need for that ATM
> pricing
Probably worth a whole separate blog TBH. It would be cheaper than Kafka but more expensive than just using the built in WAL for SlateDB or OSWALD
I do appreciate that codex is open source generally, but I don’t think it matters for this class of issue as the model is closed still
I am very excited for object storage first systems like this to leverage low latency zonal storage for write ahead logs to keep the disaggregated storage but greatly reduce write latency. That ends up being more expensive, but is likely a good tradeoff in lots of cases I have seen
I think the surprising thing is I expect flash to be a pure distillation and strictly worse quality but clearly it’s more nuanced than that.
If true, it will only be replaced with something else. With what is anyone’s guess.
I encourage you for you dev command to work with frontend setups like vite so it’s unified with backends and still supports HMR
Cool stuff thanks for sharing!
Man where in the LLM training data did the “production ready “ come from? That whole list screams AI. Humans want social proof, not a list.
https://moment-timeseries-foundation-model.github.io/
https://arxiv.org/abs/2403.07815
A friend at work used one to predict when our CEO would post in Slack, which is verry entertaining to see if correct.
I mean I also think this move doesn’t make sense, but I always find these type of comments interesting. Do people think they could do better in Mark’s shoes?