HNHacker News
TopNewBestAskShowJobs

fearthetelomere

56 karma · joined February 11, 2021

submissionscomments
fearthetelomere··on Shift-to-Middle Array: A Faster Alternative to Std:Deque?
>- 2 is not the best growth factor because it makes it harder for the memory allocator to reuse memory. More modern implementations use smaller growth factors: https://en.wikipedia.org/wiki/Dynamic_array#Growth_factor

I've often wondered about this, so I'm curious to learn more. I agree in principle we should be more clever to ensure we have better memory use. However, half the implementations in the list you've linked use a growth factor of 2, so I'm confused about your point.

If it's not the best, what is? Do you know why these implementations opt for 2 if it's not the best choice?

fearthetelomere··on Survival Analysis Part I: Basic concepts and first analyses (2003)
Survival analysis is often also called time-to-event analysis or failure analysis in engineering. In a general sense, it's used to model time until a "death" event, whether that could be divorce, machine part failure, or actually a person's death.

While methods like linear or polynomial regression can be used predict continuous variables like number of patient deaths over time, survival analysis gives us more context and allows us to make inferences about the survival of individuals and populations as a whole, in addition to forecasting illness.

It does this in a few ways, but firstly by giving us a more rich language to discuss survival data. This is where concepts like "censoring", etc. come into play. It should be noted that censoring here does not refer to the colloquial use of the word, but rather the statistical definition [1].

Survival analysis in practice includes training KM-curves, Cox-PH models, etc. by fitting them to the underlying survival data based on this censored data. It's very possible that new and more robust methods have come out since I last touched the subject around 2015. Modern NNs may blow these models out of the water in terms of pure predictive power, but from a population-modeling perspective survival analysis would likely still be very useful.

[1] https://en.wikipedia.org/wiki/Censoring_(statistics)

fearthetelomere··on Discord Reduced WebSocket Traffic by 40%
Thanks, explains a lot. Wish the article did a better job explaining that instead of framing it as a version upgrade to V2.
fearthetelomere··on Discord Reduced WebSocket Traffic by 40%
>Diving into the actual contents of one of those PASSIVE_UPDATE_V1 dispatches, we would send all of the channels, members, or members in voice, even if only a single element changed.

> the metrics that guided us during the [zstd experiment] revealed a surprising behavior

This feels so backwards. I'm glad that they addressed this low-hanging fruit, but I wonder why they didn't do this metrics analysis from the start, instead of during the zstd experiment.

I also wonder why they didn't just send deltas from the get-go. If PASSIVE_UPDATE_V1 was initially implemented "as a means to scale Discord servers to hundreds of thousands of users", why was this obvious optimization missed?

fearthetelomere··on Local First, Forever
I see, I didn't know that.

Why would one sell permanent ownership of a networked product? I've been seeing the rise of "lifetime" subscription levels (pay once) for SaaS products, but I'm curious as to what the actual long-term economics of it are.

If your cost of operations is not a one-time cost per user, I don't see how you can avoid a subscription model.

fearthetelomere··on Local First, Forever
Remember: it's local-first, not offline.

When users subscribe they should be purchasing a license that renews periodically. Local-first apps should validate the license (based on its expiry) by connecting to the internet to ensure users are subscribed. If they're not subscribed or they don't have internet access, they cannot use pay-walled features. You should offer a free, fair, and robust export tool for those not subscribed, though unfortunately not many companies do this.

What am I missing here? The drawback is of course when customers don't have internet access but are still valid subscribers, so they lose access to pay-walled features. One option would be to only offer annual licenses (infrequent license checking), another would be to offer a complementary 7 day license extension for the short-term until the user reconnects to the internet (grace period).

Stop giving away your value proposition for free. Get paid fairly.

fearthetelomere··on Local First, Forever
A bit of an aside, but CRDTs are not always the best approach to solving the local-first distributed consistency problem. For the specific given example of syncing files it might make sense, but I'm starting to see CRDTs used in places they don't need to be.

Where is your ground truth? How collaborative is a given resource? How are merge conflicts (or any overlapping interactions) handled? Depending on your answers, CRDTs might be the wrong tool.

Please don't forget about straightforward replicated state machines. They can be very easy to reason about and scale, although require bespoke implementations. A centralized server can validate and enforce business logic, solve merge conflicts, etc. Figma uses a centralized server because their ground truth may not be local.[1]

If you try a decentralized state machine approach the implementation is undoubtedly going to be more complex and difficult to maintain. However, depending on your data and interaction patterns, they still might be the better choice over CRDTs.

It could be argued that even for this example, two local-first clients editing the same file should not be automatically merged with a CRDT. One could make the case that the slower client should rename their file (fork it), merge any conflicts, or overwrite the file altogether. A centralized server could enforce these rules and further propagate state changes after resolution.

[1] https://www.figma.com/blog/how-figmas-multiplayer-technology...

fearthetelomere··on Leaving Rust gamedev after 3 years
Adderall?

I'm only half-joking :)

fearthetelomere··on LiveView Is Best with Svelte
This looks super promising...

I very much relate to the dropdown example, and I've found that complicated UX patterns can be extremely awkward to implement and maintain in LV.

One example from my experience that was prickly to implement in LV was graying out a chat message you sent if it didn't get acked or persisted by the channel/server for any reason.

Can't wait to try this out in my next project!

fearthetelomere··on Barcelona’s Sagrada Familia will be completed in 2026
While still a marvel in its own right, I couldn't help but feel underwhelmed when I visited.

After witnessing the architecture of both ancient and (relatively) modern Eastern/Southeastern structures, I found the Sagrada Familia lacking. It came across somewhat soulless with its bleached cavernous interior and uninspired stained glass artwork.

That came across harsher than I truly feel, but I think it's garnered some undeserved hype especially after 100 years of construction. I wonder what structures inspired Gaudi that may be lost to time, and what feelings he wanted to evoke in me that I didn't experience during my visit.

fearthetelomere··on Write for others but mostly for yourself (2022)
I often synthesize this as: to write, is to think.

It derives from David McCullough's quote, "Writing is thinking. To write well is to think clearly. That's why it's so hard."

In order to arrange the right words in the right sequence to convey the right message, you're required to organize your thinking, and also consider how it might be consumed.

To bring it back home, I suppose the same could be said of writing code.

---

As an aside, McCullough wrote my favorite biography, John Adams.

fearthetelomere··on The Billion Row Challenge (1BRC) – Step-by-Step from 71s to 1.7s
If you do find the time, I'm sure that writeup would be very valuable!
fearthetelomere··on Tools for Learning LLVM TableGen
As I know nothing about TableGen, can someone explain when you would reach for this tool? Perhaps even links to simple examples?
fearthetelomere··on Apple changed how reading books works in iOS 16
It may seem trivial, but I assure you it's valid criticism. FWIW, I've stopped using the Books app on my Ipad in favor of emailing my ebooks to my Kindle account and reading it through the Kindle app.

Why couldn't they have an option to keep the old animation? In 2022, it should be that simple, but of course...

It's not that I loved the old animation, but rather that I can't stand the current one. It's like I'm rotating a deck of cards, taking one off the top of the pile and putting it underneath. Not very pleasant to me, and also not an animation style I've seen used in any other book apps to my knowledge. I wonder why that is?

fearthetelomere··on Sorting with SIMD
>You should probably not use inline assembly in production code

What are the alternatives here? Write the assembly in a separate file and provide a FFI?

fearthetelomere··on Htmx in a Nutshell
Regarding #1, I've found that Phoenix + LiveView has been helpful for these cases. They have great form validation tools using Changesets and the `phx-change` attribute [1]. You can redact certain fields [2], and even treat certain fields like virtual fields (like password not being stored vs the password hash being stored) [3].

Of course, the (potential) drawback is in using Elixir for both front-end and back-end which may be a tough sell to a client or employer.

-----

[1]: https://blog.appsignal.com/2021/09/28/real-time-form-validat...

[2]: https://hexdocs.pm/ecto/Ecto.Schema.html#module-redacting-fi...

[3]: https://elixircasts.io/ecto-virtual-attributes

fearthetelomere··on NYT reporters had a top-down directive that tech coverage should be critical
I think it's a not-so-rude awakening to follow individuals rather than organizations. For example, Matt Levine could be the one you trust for finance, or Jason Schreier for news on video games.

It's potentially easier to understand an individual's biases on a per-article basis than something like the NYT, especially if you follow them and their perspectives over time.

fearthetelomere··on It wasn't for nothing
Once you find the things that anchor you physically, it makes it much easier to sail where you want to sail and stay when you want to stay. No matter what passes around me or what happens in life, having these anchors about myself lifts the good times to be more enjoyable and the bad that much more surmountable.

I'm happy for you.

fearthetelomere··on The Amazon Empire Strikes Back
Most people I talk to don't know AWS has a managed blockchain product, quantum computing service, or an analytics suite for financial analysts and traders.

They seem to have their fingers in every pie, and people don't seem to really notice or care until it's already big or too late.

fearthetelomere··on A Project of One’s Own
I had no idea that she still holds such a distinction, thanks for pointing that out! I'm always in awe of their accomplishments.
fearthetelomere··on A Project of One’s Own
There's a working theory that you could do this with anything. A cool story to read is the Polgar family. His daughters became some of the best chess players of their time.

>Polgár and his wife considered various possible subjects in which to drill their children, "including mathematics and foreign languages," but they settled on chess. "We could do the same thing with any subject, if you start early, spend lots of time and give great love to that one subject," Klara later explained. "But we chose chess. Chess is very objective and easy to measure."[3] His eldest daughter Susan described chess as having been her own choice: "Yes, he could have put us in any field, but it was I who chose chess as a four-year-old... I liked the chessmen; they were toys for me." [1]

[1] https://en.wikipedia.org/wiki/L%C3%A1szl%C3%B3_Polg%C3%A1r

fearthetelomere··on The Revolution in Classic Tetris
I didn't really realize what the quote was talking about until you mentioned mario party. That brought back memories.

Looking at the sibling comments, it's kind of cool how a lot of people arrived at the same pattern intuitively.

fearthetelomere··on “Is Parallel Programming Hard, and, If So, What Can You Do About It?” v2 Is Out
Thanks for the thorough explanation!
fearthetelomere··on Aeron: Low-latency, reliable networking over UDP
I don't think there's any official benchmarks, but I did find that someone benchmarked it 5 years ago [1]. Aeron also provides a benchmarking tool [2].

[1] https://github.com/benalexau/rpc-bench/tree/master/results/2...

[2] https://github.com/real-logic/benchmarks

fearthetelomere··on “Is Parallel Programming Hard, and, If So, What Can You Do About It?” v2 Is Out
- "A high-performance implementation would of course use padding or special alignment directives to avoid false sharing."

- "Cache alignment and padding often improves performance by reducing false sharing."

- "Software can use the alignment directives available in many compilers to avoid false sharing, and adding such directives is a common step in tuning parallel software."

There's just essentially two or three lines in the book that I could find that refer to optimizing cache use through padding/alignment, and that was after knowing what to search for. Although I wouldn't be surprised if I missed something.

In my experience, the discourse stops at the surface level, which makes the topic appear like it's obvious or trivial. But there are many follow-up questions that naturally arise for me:

- What are the trade-offs of cache-padding shared data? Why does it degrade performance for certain problems?

- What is a good rule-of-thumb for when to prioritize cache padding/alignment over cache locality?

- Are there other best practices like cache padding/alignment/locality that improve performance?

- What is an alignment directive and how does one use it?

I agree with what you've said, I was merely pointing out that I wish parallel programming resources delved more into this subject, as I feel that it's a practical and common issue. I'm sure it's not trivial and requires a fair bit of expertise, but that's why one would reach for a book like this after all.

fearthetelomere··on “Is Parallel Programming Hard, and, If So, What Can You Do About It?” v2 Is Out
One of the biggest issues I encounter with concurrency is whether the code I've written could be optimized to reduce cache misses.

I feel like a lot of resources including this wonderful book gloss over reducing cache misses. Is it really that trivial? What are the tips and tricks involved?

fearthetelomere··on Swift Evolution: Actors
>Maybe it's just me, but message flows and behavior is a lot easier to think about than concurrent access to shared state.

In general, I agree with you. I think the actor model makes concurrent code readable, understandable, and potentially easier to maintain due to that readability. People inexperienced with concurrency might find it a lot more approachable than the alternatives; an actor feels so similar to an object at times and are easier to use than threads or green-threads. I've personally found that actors also give me performance right out of the box, when I could have really screwed something up with locking.

But other than that, doesn't it all boil down to the same thing?

Actors serialize access/modification of state through ordered messages. Locks serialize access/modification of state through acquiring the lock. And when you have complex access or modification patterns in your code, the actor model doesn't help all that much. To me it's like each message acquires the lock to the actor state.

Sure, you can see what messages cause what behavior or state change for an actor A, but the rest of the system can still be complex. Which actors send a given message to actor A, and why or when do they do this? It's not obvious, and the same complexity can be found in event-driven systems, lock-y threaded systems, etc.

>With the re-entrant Swift actors proposed in the link, I'm not so sure it will be easy.

I'm personally not convinced about re-entrant actors. I don't see why you'd give up the guarantee of blocking until a message is fully handled.

They state:

"Moreover, it helps ensure that actors can make timely progress within a concurrent system, and that a particular actor does not end up unnecessarily blocked on a long-running asynchronous operation (say, downloading a file)."

This example is particularly weak in my opinion. An actor could block on the long-running async operation, of course. But isn't that the brute-force/naive approach? I can't see any one doing this in practice.

You could create a long-running actor whose only job is to sequentially handle requests to download files and send the file descriptors back as a message response. You could create a worker pool to do that as well. You could decompose your one actor into a staged pipeline of actors, so any requests that block can go to one pipeline, and other swiftly handled messages can go to another pipeline. Of course, none of these are simple solutions, but neither is designing an actor with re-entrancy in mind.

>that makes it hard to think about again

I agree. However, they do seem to be pushing the envelope and not just reinventing the wheel. I'm curious to see how this plays out.

fearthetelomere··on Swift Evolution: Actors
That's an important point, and didn't mean to make the problem of distributed actors appear trivial. In my mind, all the more reason to consider the distributed case when baking in actors into your language. For example, what distributed primitives would the language support vs leave modular for libraries?

>I've many times seen Erlang code where the developers didn't make this distinction - because the message passing operation looks the same, remote or not - and as a result the system is not resilient to network failures.

It's not about the message passing operation looking the same. The developers erroneously assumed that message delivery was guaranteed. The core issue here is not unique to actor systems.

Erlang/Elixir and Akka do not guarantee message delivery (even for a local case) from what I understand; guaranteed message delivery may mean different things in different contexts (like message queued in mailbox vs message is received from mailbox). In my opinion, developers should always program defensively when writing networked applications using something like the circuit-break pattern or making operations idempotent as you mentioned. When using actor systems, developers should not assume guaranteed message delivery unless the tool allows them to.

I'm not familiar with other actor systems so I can't speak on their guarantees, but Erlang has outlined the reasoning why messages shouldn't be considered to be guaranteed here [1].

[1] If I send a message, is it guaranteed to reach the receiver?: https://erlang.org/faq/academic.html#idp32844816

fearthetelomere··on Swift Evolution: Actors
With something like Elixir/Erlang, the distributed system is quite robust and reliable from my experience. A bit rigid and somewhat difficult to configure for custom topologies, but dependable overall.

>a plain-old set of services talking via a message-broker

That said, I think you're absolutely right with distributed Akka. I'm very hesitant to fully embrace it, and we use simple service-level APIs to communicate between nodes. I understand the developers have made a lot of progress on the functionality of remote Akka over the years, but it's just not as tried and true as I would like. Using Aeron for message transport for example, is something that may be the best tool available, but is really hard to sell to my org when simple services are more approachable and maintainable.

>actors "becoming" (the term used IIRC) other types of actors based on messages received

Yeah... I didn't understand why my org was using the Classic Akka instead of the new and improved Typed Akka for a long time. But cool-looking things like this just aren't worth it sometimes. Especially when Classic Akka "just works".

fearthetelomere··on Swift Evolution: Actors
Actors aren't just about local concurrency, and if a language is attempting to bake them in, they should leave some room for the distributed case in my opinion.

One of the biggest benefits of using actors is that whether an actor exists on the same machine or across the network can be abstracted away from you. You're just sending messages, so you can send a message over the wire and it would be as simple to the developer as sending it locally. Not considering this use case would make it a very limiting language feature.

Page 1 of 2Next →