morganherlocker

Prometheus is so easy to add and if you need more scale, there is mimir and a few other options with similar client semantics. I really can't imagine reaching for a framework APK that tries to anticipate every possible thing I would want telemtered, and is inevitably missing all the domain specific derived channels I need. Even prepackaged Prometheus exporters are usually overkill.

I think the big miss is that the former is achievable far more often than people imagine, which leads to settling for the later, aka pessimization. Reducing your allocations from millions per run to handfuls per run during initialization is effectively a "go fast" button, and is generally more reliable. It is very common for teams to spend big effort getting a 2-3x speedup on allocation-heavy code by disabling branches when a 100-1000x speedup can be had if restructuring allocations is a strategy under consideration (even before the extra 4-8x you might see if you go all the way to hand-tuned SIMD).

Disapprove and ask for a call where the author must verbally explain the changes to receive approval? This seems like a solvable problem, and one that already existed in repos with many contributors of varying skill (open source, bigco with lots of interns). Letting bugs through is an even bigger time sink.

You can setup most PR systems to squash on merge using the first commit's message, then enforce that the top commit message is prefixed with a ticket ID. This practice has many benefits: readable git log, easier git bisect for tracking down regressions, it becomes easy to find all commits associated with a block of work, more useful git blame.

> If you have a non-insignificant amount of data points to track this is going to eat just a ton of memory while also being pretty slow to encode/decode.

This is a fair critique, however, for any large GeoJSON, the coordinate arrays will dominate the size. I think it's also safe to assume this data will be gzipped at rest and over the wire, which will eliminate most of the "header" metadata size you mention. As you point out, it would be much more efficient to have a binary format, and there are good examples like these, that are ~2-3x smaller in benchmarks:

https://flatgeobuf.org/ https://github.com/mapbox/geobuf

That said, I think GeoJSON should be compared against other human readable formats like KML, which has a lot of wasted space as well, while being more difficult to read/write.

IDK, it seems like old reddit did just fine without even trying that hard. The DOM and surrounding JS API is already a high level GUI framework, and the post illustrates that it's perfectly capable of doing useful interactions >60fps. I personally love working with native code, but the modern browser is capable of producing fast interfaces that saturate human senses without it. If you write JS like a C programmer would, it will usually be quite fast, even if it's not optimal. If you write native apps in C++ like a modern JS programmer - frameworks and deps with abandon - it will be a stuttery mess.

When the DOM is not enough, there's already WebGL and WASM. A vanishingly small sliver of use cases can't saturate human senses with these tools, and the slowest, jankiest websites tend to be the least deserving of them (ie: why is jira slow? It's literally a text box with a handful of buttons on the side!).

I've used duckdb a lot at this scale, and I would not expect something like this to take more than a few seconds, if that. The only slow duckdb queries I have encountered either involve complex joins or glob across many files.

I designed some parts for an enclosure over the weekend using claude opus to generate an OpenSCAD file, which I then exported to STL and sent to a 3D printer. I was able to get a visually-correct-enough STL off to the 3D printer in about 5 minutes.

I then waited about an hour for the print to finish, only to discover I wanted to make some adjustments. While I was able to iterate a couple times, I quickly realized that there were certain structures that were difficult to describe precisely enough without serious time spent on wording and deciding what to specify. It got harder as the complexity of the object grew, since one constraint affects another.

In the end, I switched to FreeCAD and did it by hand.

- notepad++: 56.4 MB (went gray-window unresponsive for 10 seconds when opening the explorer)

- notepad.exe: 54.3 MB

- emacs: 15.2 MB

- vim: 5.5MB

I would argue that notepad++ is not really comparable to VSCode, and that VSCode is closer to an IDE, especially given the context of this thread. TUIs are not offering a similar GUI app experience, but vim serves as a nice baseline.

I think that when people dump on electron, they are picturing an alternative implementation like win32 or Qt that offers a similar UI-driven experience. I'm using this benchmark, because its the most common critique I read with respect to electron when these are suggested.

It is obviously possible to beat a browser-wrapper with a native implementation. I'm simply observing that this doesn't actually happen in a typical modern C++ GUI app, where the dependency bloat and memory management is often even worse.

> If you want electron app that doesn't lag terribly

My experience with VS Code is that it has no perceptible lag, except maybe 500ms on startup. I don't doubt people experience this, but I think it comes down to which extensions you enable, and many people enable lots of heavy language extensions of questionable quality. I also use Visual Studio for Windows builds on C++ projects, and it is pretty jank by comparison, both in terms of UI design and resource usage.

I just opened up a relatively small project (my blog repo, which has 175 MB of static content) in both editors and here's the cold start memory usage without opening any files:

- Visual Studio Code: 589.4 MB

- Visual Studio 2022: 732.6 MB

update:

I see a lot of love for Jetbrains in this thread, so I also tried the same test in Android Studio: 1.69 GB!

I frequently use a docker-compose template with prometheus pushgateway + grafana for deploying on single node servers, as described at the start of the article. It works well and is trivial to setup, but the complexity explodes once your metric volume or cardinality requires more scale like prometheus alternatives a la mimir.

I think this would not need to be an issue as frequently if prometheus had a more efficient publish/scraping mechanism. iirc there was once a protobuf metric format that was dropped, and now there is just the text format. While it wouldn't handle billions of unique labels like mimir, a compact binary metric format could certainly allow for millions at reasonable resolution instead of wasting all that scale potential on repeated name strings. I should be able to push or expose a bulk blob all at once with ordered labels or at least raw int keys.

"Everything" is another that puts the default search to shame. I've also seen people who just have a script that pumps all new files into a txt file every so often and runs bruteforce ripgrep on it, which gives instant interactive results. It's really hard to imagine coming up with a search routine that is as slow and unreliable as what ships with mainstream OS file managers.

I hear this a lot, but can you really say that you're consistently saturating a 1Gbps line for netcode or 6+ Gbps nvme for disk data? In my experience this doesn't really happen with code that isn't intentionally designed to minimize unnecessary work.

A lot of slow parsing tends to get grouped in with io, and this is where python can be most limiting.

There's currently talk of adding gigawatts of data center capacity to the grid just for use cases where python dominates development. While a lot of that will be compiled into optimized kernels on CPU or GPU, it only takes a little bit of 1000x slower code to add up to a significant chunk of processing time at training or inference time.

It shouldn't take any noticable power/cycles to accomplish this task. Having flags for "performance" littered through the codebase and UI is a classic failure mode that leads to a janky slow base performance. "Do always and inhibit when not needed".

> However those tools do not have the polish that ESRI kit does, but at leas you’re not paying the licensing!

Arguably they have more polish, but less GUI. The open GIS world is more CLI, database, and library driven, which can be an advantage for many users trying to build high reliability or scaling systems.

> so now the main problem is building the hardware, there are a lot of solutions for the software part.

While cool and all, this type of sim is a tiny, tiny slice of the software stack, and not the most difficult by a long shot. For one, you need software to control the actual hardware, that runs on said hardware's specific CPU(s) stack AND in sim (making an off the shelf sim a lot less useful). Orbital/newtonian physics are not trivial to implement, but they are relatively simple compared to the software that handles integration with physical components, telemetry, command, alerting, path optimization, etc. etc. The phrase "reality has a surprising amount of detail" applies here - it takes a lot of software to model complex hardware correctly, and even more to control it safely.

As a point of comparison, I just downloaded the Windows binary for duckdb (which provides a nice TUI for similar tasks) and it was 9.84MB. People can and should expect better.

Now instead of seeing millions of lines of inscrutable code in your program bloating binary sizes, you can see it in every program (that doesn't disable stdlib).

All of them? It should feel instant. A UI does not benefit from artificial delays.

I use syncthing. Works flawlessly and took just a few minutes to setup.

Agree. As an example, this proto generates 584 lines of C++, links to 173k lines of dependencies, and generates a 21Kb object file, even before adding grpc:

syntax = "proto3"; message LonLat { float lon = 1; float lat = 2; }

Looking through the generated headers, they are full of autogenerated slop with loads of dependencies, all to read a struct with 2 primitive fields. For a real monorepo, this adds up quickly.

If most data between the backend/frontend is raw numeric array binary blobs, and you target array types that can be loaded directly into JS TypedArrays (int32_t* -> Int32Array, float* -> Float32Array, etc.), you have a more or less optimal data transfer mechanism with zero dependencies. The JS parsing is faster than the fetching, even with 10s of millions of items, provided you keep the number of arrays fixed and reasonably small.

This is a natural fit for data-oriented programming in most systems languages, including C++. It's also compatible with numpy and any other dynamic language backends that have a performance-oriented array primitive or lib. The data can be served dynamically from memory, or written to files on a static server like nginx.

Same, and I've found this works quite well with a C++ backend. Just send over binary data as struct-of-arrays, load it into typed arrays, render components. Your app will be bottlenecked only on bandwidth, as it should be, even for heavy 3d applications.

There's nothing fundamentally stopping react apps from being fast like this, but the dependency snowball has a way of hiding accumulated dynamic memory allocations and event listeners. Instead, we should be intentionally grouping allocations and events, keeping them contained to a few well understood pieces of code. Without this, these apps could be ported to native C++ or rust and they would still be nearly as slow because there is no magic language that can make a slop of millions of tiny dynamic allocations fast and low overhead.

If you're type 1 diabetic, accuracy is paramount, especially in a closed loop insulin pump setup. Even with access to the bloodstream, existing CGMs leave plenty of room for improvement that would directly improve quality of life a lot more than the annoyance of applying the sensor under the skin.

Everything listed here is a great pitch for vanillajs with zero framework at all.

> Shaker said. “No other technology can provide this level of precision without direct contact with the bloodstream.”

The existing alternatives do have access to the bloodstream.

> (There are ECS frameworks in JavaScript, which gives you absolutely no control over memory layout and thus completely defeats one of the primary purposes of the pattern.)

While JS does not provide great support for bit packing complex structs, typed arrays give you quite a bit of control over memory layout for simple numeric types, which is what you usually want for optimal data-oriented code anyway. This is a common technique used in fast JS libs for data visualization, ie:

https://github.com/mourner/flatbush

There are also basic operators required for bitarrays, which are useful for ECS and memory-efficient code generally.

My personal homelab (a stack of old laptops connected to a network switch) is mostly built around various experimental antenna arrays used for rtl-sdr hobbies (aircraft and maritime telemetry collection mostly) + home automation over a zwave mesh network. The home automation ecosystem also has a lot overlap with automated gardening/growing, since automated sensing and irrigation are a great use case for tools like home assistant.

> A large number of people who live in the project are people on section 8 and disability with no ties to the region whatsoever.

No ties besides… living in said community. How is the disability status of residents relevant to your classification of the project as a failure? Are you advocating for federal housing aid to come with requirements like “locals only” and “non-disabled residents only”?