This problem is larger than crypto. It seems to be related to the accessibility of trading to "retail investors" (enabled by lower fees, new technology, monetary policy, etc.).
I'm surprised so many people on Hacker News apparently still believe that you can "see all transactions" on every blockchain. It really only takes some surface level research to find working, production blockchains/cryptocurrencies that use one of several types of cryptography to hide that data, plus if you're using a pseudonymous blockchain, non-custodial ways to "break the link" between wallets.
Most have no knowledge of the state of the industry. Their opposition is based on knee jerk reactions.
Are any of those even in the top 30 cryptocurrencies by market cap? Is anyone speaking about them in the context of Web3 or is it all ETH, XRP and maybe two more?
From my blissfully uninformed mind, the only time these cryptocurrencies pop up is precisely this: as a counterargument to the popular ones, where you can see all the transactions. They're fringes even among the fringe that is cryptocurrencies.
Aztec Network, Tornado Cash, and Starkware all run on top of Ethereum... Aztec and Starkware are currently in the community spotlight because they're rollups or so-called "Layer 2s" that push the state of the art of Ethereum's tech (and the state of the art in zero knowledge cryptography, especially Starkware's CAIRO language and their EVM to zero knowledge proof circuit transpiler). Tornado Cash has over $4B in deposits, and for many is the go-to way to anonymize transactions on Ethereum.
Ernst & Young released Baseline last year, a zero-knowledge cryptography toolkit for operating private data services on the public Ethereum network.
The answer to "who cares" is basically the entire nascent industry, plus a handful of mathematicians and cryptographers - if you are paying attention.
A number of the top 10 networks will eventually (~5-10 year time frame) get privacy preserving transactions however the space is still very much in development and it's a bit easier to reason about the network and what changes to make next when you can see what the users are doing. I know at the very least, IOG (the company currently leading Cardano development) is doing privacy preserving smart contracts research with the intent that it eventually becomes part of the protocol.
There's a lot for Layer 1 protocols to accomplish as it is so the space is largely delegating privacy chain research/functionality to L2s for the time being.
Monero uses a system that at a high level is referred to as Ring Signatures (it's not too complicated but it's more than I want to get into at the moment). Zcash and other networks use zero knowledge non-interactive proofs to prove consistency/correctness.
The major drawback of privacy preserving chains is that the transactions are expensive to compute due to all the maths that goes into maintaining that consistency without publicly leaking information. This isn't prohibitive for a desktop but a mobile phone will likely take a few minutes to build and send a transaction in the current environment (can be improved by HW acceleration or general HW improvements).
A nullifier is derived deterministically from private account data, and zero-knowledge proofs are used to show that a nullifier was dervied correctly without revealing the private account data.
For more detail I think this is a good explanation: https://electriccoin.co/blog/zcash-private-transactions/
Here's a link to the Ring CT papers. The original explains the base system, 2.0 formalises the security of the system, and 3.0 describes how the original protocols had issues and how they were improved.
Original Ring CT Paper: https://eprint.iacr.org/2015/1098
Ring CT 2.0: https://eprint.iacr.org/2017/921
Ring CT 3.0: https://eprint.iacr.org/2019/508
Personally I find the Ring CT 3.0 paper to be the most enlightening of these. For a simpler/shorter explanation however, Moneropedia has some videos that are helpful in this.
https://www.getmonero.org/resources/moneropedia/ringsignatur...
https://www.getmonero.org/resources/moneropedia/ringCT.html
-------
The key details are these.
Monero uses a UTxO (unspent transaction outputs) accounting model. There are a number of networks that use this model, namely Bitcoin. The UTxO model doesn't strictly have a concept of accounts but rather keys and UTxO. You can think of UTxO as as atomic chunks of data/value. These UTxO are created once and used once. If you want to spend money, your inputs are UTxO, the outputs are UTxO, and the balances of all of these inputs and outputs must sum up to a net 0. Now UTxO can only be spent if a valid cryptographic signature can be produced. If your private keys can produce that signature, the value held in that UTxO can be spent by you.
This gives us our three key points. All UTxO consumed or spent in a Tx must collectively sum up to zero, UTxO can only be spent if the proof object/cryptographic signature produced is valid, and UTxO can only be spent if they haven't yet been spent.
First Ring CT has each sender derive a one time private key from their private key and a one time public key derived from the recipient's public key. These keys are used to produce a set of encrypted "coins" which essentially hold the value of the UTxOs being spent and then created. This allows the sender and recipient to know the contents of the actual meaningful transaction. Then a cryptographic proof verifies that out of all the public keys in the Tx, all the keys can spend their selection of coins included in the Tx and only one of them is the real set of coins. From there the balance is verified by doing that some clever math where the sum of the encrypted outputs is divided by the sum of the encrypted inputs must equal some constant determined by the encryption key. This is a bit of a simplification but it works out such that it is equivalent to "ins - outs = 0" in a traditional UTxO system.
The key/signature verification mentioned previously has a nifty property where the produced "key images" serve as what are effectively "hashes" of the UTxO. This means that if any key image has previously been used in a transaction, it effectively guarantees that the corresponding UTxO has already been spent. Going back and checking this is somewhat more expensive than on a traditional network however optimisations (like using a bloom filter) allow for this check to run significantly faster.
That's how Monero upholds those 3 properties that are required for consistency in a UTxO based network without leaking information about the sender, recipient, or balances. In short it's using one time PKI derivations to break information symmetry and then from there structuring the encrypted data so that you can use some clever arithmetic and signature checks to verify correctness.
Apologies if my explanation isn't terribly clear or perfectly accurate. It's my reasonably solid understanding of the system. I'm not a formally trained cryptographer or mathematician so my choice of words or explanations may be somewhat inaccurate however they give a reasonable coverage of how the system works. If you want to dive more into the meat of it, I'd seriously recommend the Ring CT 3.0 paper as it's really well put together once you can get through the terse notation and dense amount of information.
You already know they are sharding secrets into nodes with Intel encryption co-processors but if you need more detail about how you got to hit them up