mightybyte

I think the general concept here is putting in place restrictions on what code can do in service of making software more reliable and maintainable. The analogy I like to use is construction. If buildings were built like software, you'd see things like a light switch in the penthouse accidentally flushing a toilet in the basement. Bugs like that don't typically happen in construction because the laws of physics impose serious limitations on how physical objects can interact with each other. The best tools I have found to create meaningful limitations on code are a modern strong static type system with type inference and pure functions...i.e. being able to delineate which functions have side effects and which don't. These two features combine nicely to allow you to create systems where the type system gives you fine-grained control over the type of side effects that you allow. It's really powerful and allows the enforcement of all kinds of useful code invariants.

Here's a long personal interview with Durov.

https://x.com/TuckerCarlson/status/1780355490964283565

I know that TuckerCarlson is a polarizing character. My posting of this link is not any kind of statement for or against him or his politics. That being said, the interview really gives an interesting picture of Pavel Durov IMO. If you can ignore Carlson's annoying tangents into American politics, you get to hear a good bit of Durov's life story straight from his mouth in reasonable detail. I came away from it with a more positive picture of Durov and Telegram.

Ahh ok. I think my source was someone else, but it sounds like they were probably citing Feynman. Good to know.

I really like an expansion of this idea that I heard somewhere awhile back:

Keep a few significant problems in your mind...and also keep a few significant solutions / problem solving techniques in your mind. Then when you encounter new problems, check them against your set of solutions and see if any of them apply. Also, when you encounter new problem solving techniques, check them against your set of problems to see if they're applicable. Whenever you encounter a new problem or solution that seems unusually significant, add it to the list that you keep track of

Not endopasta?

Yes, this post is classic. My answer to Brent's "monad tutorial fallacy" is https://mightybyte.github.io/monad-challenges/. It was inspired by The Matasano Crypto Challenges that Thomas Ptacek & others created awhile back which, instead of trying to teach you cryptanalysis, guides you down the path of actually doing realistic cryptanalysis with a series of challenges.

The book "Against Intellectual Monopoly" by Michele Boldrin and David K. Levine has a decent overview of the arguments supporting this claim.

https://annas-archive.org/search?q=against+intellectual+mono...

Yes, I have similar thoughts. Getting older, traveling more, seeing the hustle and bustle of large cities, becoming more aware of all these details that many people don't think about, etc...has all greatly expanded my world view. One of the big thoughts that these kinds of realizations have left me with is how vanishingly small each human's view is compared to the total scope of the universe. None of us is able to escape the confines of our skin and our personal perspective. However, we all tend to extrapolate this vanishingly small slice of ours to the whole universe. Problem is, it's very difficult to figure when that extrapolation is valid and when it is over-fitting, which is probably the root of a lot of human conflict.

Ooh very nice, thanks for the tip!

Your command line solution doesn't give quite the same result as OP. The final output in OP is sorted by the count field, but your command line incantation doesn't do that. One might respond that all you need to do is add a second "| sort" at the end, but that doesn't quite do it either. That will use string sorting instead of proper numeric sorting. In this example with only three output rows it's not an issue. But with larger amounts of data it will become a problem.

Your fundamental point about the power of basic shell tools is still completely valid. But if I could attempt to summarize OP's point, I think it would be that SQL is more powerful than ad-hoc jq incantations. And in this case, I tend to agree with OP. I've made substantial use of jq and yq over the course of years, as well as other tools for CSVs and other data formats. But every time I reach for them I have to spend a lot of time hunting the docs for just the right syntax to attack my specific problem. I know jq's paradigm draws from functional programming concepts and I have plenty of personal experience with functional programming, but the syntax and still feel very ad hoc and clunky.

Modern OLAP DB tools like duckdb, clickhouse, etc that provide really nice ways to get all kinds of data formats into and out of a SQL environment seem dramatically more powerful to me. Then when you add the power of all the basic shell tools on top of that, I think you get a much more powerful combination.

I like this example from the clickhouse-local documentation:

  $ ps aux | tail -n +2 | awk '{ printf("%s\t%s\n", $1, $4) }' \
      | clickhouse-local --structure "user String, mem Float64" \
          --query "SELECT user, round(sum(mem), 2) as memTotal
            FROM table GROUP BY user ORDER BY memTotal DESC FORMAT Pretty"

I'll second this. Clickhouse is amazing. I was actually using it today to query some CSV files. I had to refresh my memory on the syntax so if anyone is interested:

  clickhouse local -q "SELECT foo, sum(bar) FROM file('foobar.csv', CSV) GROUP BY foo FORMAT Pretty"

Way easier than opening in Excel and creating a pivot table which was my previous workflow.

Here's a list of the different input and output formats that it supports.

https://clickhouse.com/docs/en/interfaces/formats

It allows you to separate non-stateful business logic from the stateful update logic. Reactive frameworks usually handle the state mutation for you, eliminating the potential for UI glitches where something that depends on the updated state doesn't get recomputed/redrawn.

There are many situations where the extra constraints on pure functions are very useful. Pure functions can't segfault. They can't overwrite memory used by other parts of your program due to stray pointer references. They can't phone home to Google. Verification and optimization is often much simpler when you know that some functions are pure. Reactive programming frameworks that execute callbacks in response to events can cause really weird bugs if the callback mutates state somewhere. I could go on and on. Nobody is arguing for pure functions all the way down. A program's top-level functions will usually be side-effecting, and then you can call out to pure code any time. In practice, a surprising amount of application logic can end up being pure functions.

I also use plain text files for a lot of my personal organization. My system isn't quite like what OP describes, but it has some things in common. Some of my files are also structured by date as a never-ending journal. This isn't for a todo list, it's for a journal of things I encounter that I'd like to be able to find again and that I don't want to accumulate as clutter elsewhere...i.e. in browser tabs, etc. Sometimes it's a web link, or maybe something I learned somewhere, something someone told me, etc. I include notes whatever words / strings I think I might use if I want to find this particular thing later. I use org mode and make each date be a top-level bullet so I can nicely leverage powerful text search tools like ripgrep, regular expressions, etc.

I don't find it useful to force everything into a single file. Instead, I'll organize these text files somewhere inside a directory structure that I can recursively grep. Unlike the OP I do use mutable TODO lists to track high level lists of things that I want to continue to spend mental energy on, but I do like the chronological list of done things and I might think about adding something like that or maybe augmenting the chronological notes file I already have.

I do depart from the world of plain text for keeping track of larger amounts of information such as good papers I encounter, complete blog posts that I might want to refer back to, etc. For this I use the fantastic DEVONthink tool. It's got a large array of powerful features including automatic OCR and indexing of images and an excellent search feature, but the one that I use the most is its ability to make a "web archive" from a link. This downloads all of a web page's resources and stores them in the database locally, making it really easy to refer back to things that I've seen before regardless of whether I have internet access or not, whether the website is still around, etc.

Funny you mention that. I do that too, except in my case it's Spacemacs in vim mode. I do go back to them from time to time, but that's mainly because it's an easy ripgrep search of the whole notes directory tree. If I didn't have search and was only using the editor to look things up, I might never refer back to the old notes.

I came to what I think is a related conclusion fairly recently (https://twitter.com/mightybyte/status/1749513355038220653): One of (if not the) most valuable skills in life is being able to choose that you're going to consider an arbitrary thing fun.

Your good mood skill seems like the interpersonal version of that. It's always fun to see ideas from different areas form a coherent big picture.

That volume is not wash trading (i.e. "fake volume" in my words). There is fake volume in crypto but that happens largely on the smaller, less established exchanges that are trying to make a name for themselves. You can see this from the "**" markings on CoinMarketCap that indicate that they don't include that volume in their total volume calculations. I did not include any **'d volume in my above calculations.

But don't take it from me. This comment https://news.ycombinator.com/item?id=39200966 elsewhere in this thread agrees that you can move that amount of Bitcoin in a small number of weeks.

> All that matters is if there if fresh money coming in

Every time a trade happens, someone is "coming in" and someone is "getting out". The more trades there are, the more opportunities there are for anyone (including the German government) to take the other side.

> after the recent lack of price action from the ETFs, it doesn't suggest the existing volume is legitimate.

This is by no means the only reasonable interpretation of the price action surrounding the ETF approval. It's totally plausible that the lack of price action just means that the amount of money "selling the news" of the final ETF approval is around the same as the amount of money "buying the rumor" of a big influx of cash.

Those numbers aren't new inflows of capital into Bitcoin. They're the total volume traded both buys and sells. There are traders, market makers, etc both buying and selling many times a day, and anyone wanting to make large block trades can get a sufficiently modest piece of that. 5% is very achievable with very little price impact even in markets with lower liquidity than Bitcoin.

A quick look at the 24-hour trading volumes of the top 10 crypto exchanges (only a single trading pair per exchange of BTC against a USD or equivalent) gives a total volume of ~3.8 billion dollars. Many people say that there is a lot of fake volume in crypto but this is still a pretty conservative number. The total BTC volume according to CoinMarketCap is about 23 billion, but I prefer the smaller number because it gives a much better picture of the volume that could be feasibly captured with pretty unremarkable exchange access. If we assume that one can capture 5% of this 24-hour volume (a reasonably conservative assumption given good enough trading infrastructure), it would take about 11 days to offload that amount of bitcoin by selling across the top 10 exchanges.

So you absolutely could convert an amount like that into "real" assets. 11 days doesn't seem like an unreasonable amount of time to me, and we haven't even talked about OTC deals or other providers offering block trading services.

In short, the government is special because of its broad sweeping power to make laws that impact everyone. For example, consider the first amendment: "Congress shall make no law...abridging the freedom of speech". This applies to the government but not, say, to a third grade classroom rule against using curse words. It's perfectly reasonable for a private group to ban certain kinds of speech because if you don't like it, you can go somewhere else. But much more care must be taken when you're operating in the sphere of laws and government actions.

Another way to put it...the government is the only entity in society with a monopoly on the use of force. With great power there should also be a great degree of responsibility.

Oh yeah, memory is not perfect, and I think that very often you'll have to work with a concept multiple times to fully absorb it. I think we're saying very similar things.

One person who stands out in my mind for interviewing skill and quality is Sean Evans on Hot Ones. The whole show is a great combination of excellent background research, interviewer preparation, empathy, and the setting of ultra spicy food that makes the guest very vulnerable. These factors combine unusually well to increase the effectiveness of the whole thing.

> Read the documentation. Don't skip over it to the part you want; read the whole thing, cover to cover.

I think there are times when this works and there are times when it will be a huge waste of time. In general, I think it's often very hard to tell which approach will be more efficient for you. As some sibling comments have mentioned, I think the way each person's individual brain works is a significant factor, but I think there are other potentially unknowable factors that are significant as well.

The best thing I've come up with to deal with this is to use an iterative-deepening-like approach. There's a reason this algorithm (pre AlphaGo) was the most common approach for many game playing programs. The general idea is to go a ways down a particular path but always keep in mind some notion of the global suitability of this path and when it starts to look too hard, back up and investigate some other approaches to at least a shallow (but a little bit deeper than before) depth. This lets you avoid potentially costly dead-ends for relatively low overhead.

(These thoughts inspired from this nice talk: https://www.youtube.com/watch?v=Z8KcCU-p8QA)

Can you give more detail about your self-hosted storage solution? I've wanted something like that for awhile.

I don't think that's the way it works out in practice. The fact of the matter is that deadlines are missed all the time. In many cases, there is no such thing as 100% certainty that you'll hit a "deadline"--there are always circumstances outside your control (global pandemics anyone?). There's just some implicit confidence threshold or other assumptions lurking around that probably need to be communicated. Do you want three 9s of confidence? Five 9s? Those things are very different and the cost to actually achieve the latter can often be prohibitive. Everyone benefits if we make explicit our pre-conceived idea of precisely what "cannot exceed" means.

Completely agree with this idea. And I would add a corollary...date estimates (i.e. deadlines) should also have error bars. After all, a date is a forecast. If a stakeholder asks for a date, they should also specify what kind of error bars they're looking for. A raw date with no estimate of uncertainty is meaningless. And correspondingly, if an engineer is giving a date to some other stakeholder, they should include some kind of uncertainty estimate with it. There's a huge difference between saying that something will be done by X date with 90% confidence versus three nines confidence.

Exceptionally well stated. This agrees with my experience as well.

This is an interesting idea. For me the magical period of the internet was also the nineties to mid-naughts. But I'm not sure. It doesn't seem entirely age-relative to me. It seems like something changed. I really have no way to say for sure. In fact, I was recently talking to someone a couple decades or so younger about similar topics and he indeed seemed to have more of the age-relative view. I plan on talking to him more at some point to get a better understanding of his view. But I don't think the two ideas are mutually exclusive. I'm sure many people have a magical period of wonder where their world is expanding. To be perfectly honest, in some ways I've continued to have that even in recent years. There's a really cool corner of YouTube that has tons of incredible content that IMO is revolutionizing education. But I also feel that as a species we humans haven't figured out how to handle the powers of communication that the internet has made available to us. In an age of unprecedented access to the world's information, misinformation still abounds (no matter which side of the political aisle you happen to be on). I don't know that anyone has any really compelling ideas about how to deal with this, but I think it's a significant issue that we all collectively need to work on. The question is, will we be able to come together and do so or have we already been irrevocably torn too far apart?

That seems reasonable to me. But I guess therein lies the rub. I get impression that many BH shareholders feel Warren and Charlie will be better at deploying that capital than they will.