501 karma · joined November 25, 2014
what do you mean by this ? do you rewrite the context in your proxy ?
One thing I'm curious about, is there a way to add a bookmark from a browser on iPhone ? When I'm browsing I often want to save an article in my reading list, right now I'm using Notion with the Share feature. Curious if you had the same need.
The wording « We help you setup Firecracker VMs » made me think it was a SaaS at first but maybe I was mistaken. Is the idea that a community should host its own instance of shellmates ?
I do have some questions: can the owner of a server split it any way they want ? For example if I have a 64 GB server with 16 cores (something OVH provides IIRC) can I split it into 32 2GB/0.5CPU VM ? (assuming firecracker can handle fractions of a CPU, I don’t know). This could make for some really cheap VM compared to even OVH’s offering.
For that matter do you only support certain types of server or can the owner choose any server they want from Hetzner or OVH ?
But I did write an importer for the csv files my bank provides and with smart_importer I don’t even have to categorize the statements anymore (although there are mistakes sometimes). I don’t gather metrics though , I use fava to have a visual view of my books.
I usually spend half an hour per month maintaining the ledger .
I don't use my car that often but when I do I want to be able to access my music library. Right now I'm stuck with Plex and Prism on iOS because other solutions are not good in that regard as far as I've seen in my testing.
Software wise, I use Fedora Server, although if I had to reinstall now I would use AlmaLinux because I don't need the new stuff coming with Fedora Server on my NAS.
I appreciated some things, like the fact that the syntax is close to Go, the easy interop with C, the fast compilation, somewhat complete stdlib.
Other things I didn't like much: pointers have changed to '^', I'm sure there's a reasonable but I just don't like it, I'm used to '*'. Error handling is a PITA as far as I can remember. Meta programming felt clunky.
This is all subjective based on my limited experience, but I felt that Zig worked better for me.
Configuration complexity: there are a couple of things we had to tune over the years, mainly regarding the log cleaner once we started leveraging compacted topics, but other than that it's pretty much the default config. Is it the most optimal ? No but it's fast enough. Hardware choice in my opinion is not really an issue: we started on HDDs switching to SSDs later on, the cluster continued working just fine with the same configuration.
Scaling I'll grant can be a pain. We had to scale our clusters mainly for two reasons: 1) more services want to use Kafka therefore there are more topics and more data to serve. This is not that hard to scale: just add brokers to have more capacity. 2) is when you need more partitions for a topic; we had to do this a couple of times over the years and it's annoying because the default tooling to do data redistribution is bad. We ended up using a third party tool (today Cruise Control does this nicely).
Maintenance: yes, you need to monitor your stuff. Just like any other system you deploy on your own hardware. Thankfully monitoring Kafka is not _that_ hard, there are ready made solutions to export the JMX monitoring data. We use Prometheus (prometheus-jmx-exporter and node_exporter) almost since the beginning and it works fine. We're still using ZooKeeper but thankfully that's no longer necessary, I just have to say our zookeeper clusters have been rock solid over the years.
Development overheads: I really can't agree with that. Yes, the "main" ecosystem is Java based but it's not like librdkafka doesn't exist, and third party libraries are not all "sub par", that's just a mischaracterization. We use Go with sarama since 2014, recently switched to using franz-go: both work great. You do need to properly evaluate your options though (but that's part of your job). With that said, if I were to start from scratch I would absolutely suggest starting with Kafka Streams, even if your team doesn't hava java experience (I mean learning Java isn't that hard), just because it makes building a data pipeline super straightforward and handle a lot of the complexities mentioned.
* using an external tool, just like any other RDBMS (I know and use https://github.com/golang-migrate/migrate)
* embedding migrations in the app and letting the app manage migrating up.
And yes you need to make sure GOTOOLCHAIN=auto, my colleague had to explicitly set it because it defaults to local on Gentoo.
When we migrated to it we replaced 6 prometheus servers with only 1 victoria metrics server, it was crazy efficient. We ran with a single node for all our metrics for 3 years while the number of data points observed kept growing. Today we run a 7 node cluster.
I have no idea if today Prometheus could serve our workload as efficiently but for information, a provider wanted to quote us like 300k€ per year to serve the same metrics; our setup doesn't cost anywhere near that.