HNHacker News
TopNewBestAskShowJobs

r4um

10,790 karma · joined February 27, 2013

systems guy • curious and unflinching • 99.999% positive

blog: https://blog.r4um.net

[ my public key: https://keybase.io/r4um; my proof: https://keybase.io/r4um/sigs/5PMmFocRMv45OxLv3ml3Akp2y5K6VRRr13cvQCUxz-Y ]

submissionscomments
r4um··on Fundamental of Virtual Memory
A must read would be "What Every Programmer Should Know About Memory"[1], Section 4 covers Virtual Memory.

[1] https://people.freebsd.org/~lstewart/articles/cpumemory.pdf

r4um··on Entropy and Life (2013)
Another great read on the subject "Into the Cool: Energy Flow, Thermodynamics, and Life - Eric D. Schneider, Dorion Sagan" https://www.goodreads.com/en/book/show/52737
r4um··on FF4J – Feature Flags for Java
Agreed. Problem isn’t timed rollout but lack of automated rollback. If you don’t have one don’t do this. The automated rollback should look for acceptable error rates, latency etc.
r4um··on SN2023ixf is the closest supernova in a decade
Rochester academy compilation. https://www.rochesterastronomy.org/sn2023/sn2023ixf.html

Some more pictures and animations.

https://www.instagram.com/p/Csc-QilOJ7j/ https://www.instagram.com/p/CsdEX8HMEID/ https://www.instagram.com/reel/CsdmaY0g4yS/ https://www.astrobin.com/7jvafm/0/ https://www.astrobin.com/uokdzt/

r4um··on Information Dynamics and the Arrow of Time
Coming from http://www.arameb.com/blog/2022/10/19/time-presentation
r4um··on Ask HN: What is your favorite book for learning statistics?
I highly recommend The Art of Statistics.

https://www.goodreads.com/book/show/43722897-the-art-of-stat...

r4um··on Entropy Will Fuck You
A related great read is the book Into the Cool[1]. Highly recommend.

[1] https://press.uchicago.edu/ucp/books/book/chicago/I/bo353393...

r4um··on Mitre Systems Engineering Guide (2014) [pdf]
You can try the MIT one https://www.youtube.com/playlist?list=PLUl4u3cNGP60jIMmB53zl...
r4um··on Distributed consensus revised
Part 2 https://blog.acolyer.org/2019/05/08/distributed-consensus-re...
r4um··on Linux 5.0
https://kernelnewbies.org/LinuxChanges?rev1=371
r4um··on Statistics Done Wrong
Another good curation https://web.ma.utexas.edu/users/mks/statmistakes/StatisticsM...

How to Lie with Statistics by Darrell Huff https://en.wikipedia.org/wiki/How_to_Lie_with_Statistics

r4um··on Queueing theory: The science of waiting in line
+1 For the above mentioned book, especially if you are looking at computer systems/networks.

For more pragmatic treatment found "Fundamentals of Queueing Theory"[1] really good.

[1] http://mason.gmu.edu/~jshortle/fqt5th.html

r4um··on Mental Models I Find Repeatedly Useful (2016)
Farnam Street curation is also good https://fs.blog/mental-models/
r4um··on I'm Scott Aaronson, quantum computing/computational complexity researcher. AMA
How far are we from "emergence" in terms of AI ecosystem ? Will quantum computing pave the way for it ?
r4um··on Inside the Linux boot process (2006)
This is very old, for a more recent version https://github.com/0xAX/linux-insides
r4um··on Running Java in a Container
Not just the JVM, lot of libraries build thread pools based on number of processors and some of them even hard code the multipliers (X the number of processors etc).

Setting up each one becomes a lot of work fast, so we wrote a LD_PRELOAD[1] hook that overrides sysconf[1] _NC_PROCESSORS* call to get number of processors availabale/online to a certain specified value and is baked in the docker image by default during builds.

[1] http://man7.org/linux/man-pages/man3/sysconf.3.html

r4um··on February 28th DDoS Incident Report
BGP Sim during the changes https://stat.ripe.net/widget/bgplay#w.ignoreReannouncements=...
r4um··on Global secondary indexes
Spanner secondary index link is broken correct one https://cloud.google.com/spanner/docs/secondary-indexes
r4um··on How the JVM compares strings on x86 using pcmpestri
Another excellent series on JVM perf/internals https://shipilev.net/jvm-anatomy-park/
r4um··on Truly Seamless Reloads with HAProxy
This is not needed since version 1.6 See https://cbonte.github.io/haproxy-dconv/1.6/configuration.htm...

It can do DNS resolution at runtime, additionally you can override bunch of options as to how DNS resolution behaves in terms of TTL, Failures etc

r4um··on Truly Seamless Reloads with HAProxy
There is an option added in 1.7 which makes old process exit after a grace period https://cbonte.github.io/haproxy-dconv/1.7/configuration.htm...
r4um··on Some questions about Docker and rkt
Other security controls aside, for the user thing we make it mandatory to have USER directive in the Dockerfile, else the docker builds fail.
r4um··on Docker not ready for primetime
These are some good curated books https://msdn.microsoft.com/en-us/library/dn568099.aspx http://shop.oreilly.com/product/0636920023777.do

These are not necessarily container centric.

r4um··on gRPC: Internet-scale RPC framework is now 1.0
Primarily performance, HTTP HOL[1] kills latencies and wastes a lot of resources, especially with micro services spanning requests recursively to multiple services.

We have had to force keep alive and even forcefully turn it off in some cases :(.

[1]https://en.wikipedia.org/wiki/Head-of-line_blocking

r4um··on The Ethernet PAUSE frame
Yes it does, receiver side advertises window as 0 and persist timer kicks in. The send call then blocks until window recovers.
r4um··on Distributed Logging Architecture in the Container Era
We haven't dealt with those yet. But in general we are always pushing everyone to log judicially, use metrics wherever possible (numbers are still easier to manage than strings), use sentry for exceptions logging and introducing zipkin for tracing.
r4um··on Distributed Logging Architecture in the Container Era
Same for us fluentd was very resource hungry, heka brought down utilization significantly. Regarding the buffering we just added kafka in between for graylog/ES scaling/maintenance.
r4um··on Distributed Logging Architecture in the Container Era
That is the peak we see right now at ~20-30% utilization overall network/cpu wise, it can do much more easily.

> Binary logging is a must, because even something like string formatting is simply way too slow, by an order of magnitude.

true, string processing takes most of the resources.

r4um··on Distributed Logging Architecture in the Container Era
Inbuilt docker + fluentd driver + plain ES falls flat on its face under scale. Not to mention docker daemon keeps dying :(.

Whats working for us now is local docker json logs -> heka -> kafka -> graylog -> ES. We even dockerized graylog to scale the processing up and down on demand.

Does 250k messages/sec at peak easily, More details in this presentation https://www.youtube.com/watch?v=PB8dBnpaP8s

r4um··on Linux-insides: sequential locks in the Linux kernel
LWN's coverage is pretty good https://lwn.net/Articles/262464/
Page 1 of 2Next →