Opening up ‘Zero-Knowledge Proof’ technology
blog.google
blog.google
https://blog.cryptographyengineering.com/2014/11/27/zero-kno... was a good intro for interactive ZK proofs but I haven't been able to find something for non-interactive ones.
This blog post comparing ZK-STARKs to erasure coding is in the right flavor but didn't quite stick to my brain either: https://vitalik.eth.limo/general/2017/11/09/starks_part_1.ht...
A simple signature scheme is based on proof of knowledge PoK{x : pk = g^x}, which is transformed into a noninteractive variant via the Fiat-Shamir transformation, where the message is appended to the hash. Range proofs work similarly, with the simplest form being for a single bit: PoK{(b,r) : C = g^b * h^r & b(b−1)=0}. This proves that commitment C contains a bit b in {0,1} without revealing which value it is.
Arbitrary ranges can then be constructed using the homomorphic properties of commitments. For an n-bit range, this requires n individual bit proofs. Bulletproofs optimize this to O(log n) proof size, enabling practical applications.
The commitment C can be issued by a trusted third party that signs it, and the user can then prove certain properties to a service provider, such as age ranges or location zones (constructed from latitude and longitude bounds).
A key challenge is that reusing the same commitment C creates a tracking identifier, potentially compromising user privacy.
by providing a picture of waldo in the cut-out, you can prove you know where he is without providing the location. a zero knowledge proof.
Where’s Waldo as presented isn’t even a proof of knowledge
It certainly gives a "layperson" example of being able to prove you know something without revealing it, which isn't the whole definition of ZK but is the idea driving it.
Say the whole image looked like noise and was generated from quantum measurements, and the coordinates to hash for the problem were generated with quantum measurements, and you were given the silhouette and the hash of the noise within to look for. I could see it for proof of work: you could slide along a hashing window and prove you actually did work examining half the image on average or whatever.
It basically consists in the prover getting its random challenges from hashing public inputs, rather than from the verifier's coin tosses.
If I understand correctly:
* The prover commits to a starting value (public input)
* Instead of waiting for an interactive challenge, they hash it and use the resulting hash output as if it were a challenge
If we believe the hash is a random oracle (as we do for cryptographic hash functions), then it is hard for the prover to manipulate the challenges. Is that it?
That doesn't explain the way this scheme works, but it's a nice start.
https://blog.cryptographyengineering.com/2014/11/27/zero-kno...
Usually in an IP, the prover (Bob) has to answer questions from the verifier (Alice), and Alice chooses her questions by flipping a coin. If the Bob doesn’t really know the answer, he’ll get caught cheating with high probability.
So now the trick: Bob starts generates his initial answer. Then he hashes it (“commits” in the jargon), and uses the hash as “Alice’s first coin flip”. Then he answers the question for that flip, hashes the whole thing for “Alice’s second coin flip”… etc.
Bob does this say, 100 times, and then sends the whole simulated conversation to Alice. Alice can verify that he didn’t cheat by checking the intermediate hashes.
The whole thing depends on the ability to not control the result of the hash function, so it’s vital to use a cryptographically secure one.
All the proofs that I know of allow one to get lucky with probability about .5 in each round. When you do an interactive proof with 100 rounds, you have a 2^-100 chance of getting away with cheating.
When you go non-interactive with 100 rounds, an adversary could cheat by trying about 2^100 proofs. So you replace a stronger guarantee with a weaker one, but 2^100 is a pretty big barrier.
(I just looked and the Wikipedia page and it’s very confusing fwiw)
But the number of questions you need to compensate grows only a little.
For example, interactively if you ask for Merkle tree proof that selected leaf values have a particular property, you only have to ask for about k leaves to get probability 1-(2^-k) that you'd catch a dishonest prover who had committed a Merkle root with less than half the leaves having the property.
Non-interactively, a dishonest prover could secretly grind attempts, say 2^g times, and then you'd have a lower probability of catching them, approximately 1-(2^(g-k)). But g can't be all that large, so you can increase k to compensate without making the proof much larger.
You.can also require certain hashes to have a fixed prefix, like Bitcoin mining, forcing every prover to have to grind 2^p times. This reduces the effective g that a dishonest prover can achieve, allowing k to be smaller for the same security, so allowing the non-interactive proof to be smaller. At the cost of honest provers having to grind.
The non-interactive piece is pretty straightforward you just simulate challenge response conversation with unbiasible public randomness and show the transcript (Fiat Shamir transform).
Another area worth exploring is how some of these proof systems can have such incredibly small proofs (192 bytes for any computation in groth16 zk snarks). That relies on the much more difficult to intuit theory of elliptic curve pairing functions.
For example, in the usecase of providing a proof-of-age to a website: who provides the verification data (the government?); what form does that take (a file in a standard format?); who holds/owns the verification data (the user?); who runs the verification software (the end-user's web browser?).
Can the user use any implementation to provide the proof, or must it be a "blessed" implementation such as Google Wallet?
> Your phone stores the document, with cooperation from a secure element that binds the document to the phone. The website you visit verifies the proof.
So it does require a "blessed" implementation, and I have to trust Google or Apple to handle my data? I cannot own the document myself and use an open-source client that I trust to provide the proof?
In other words, does the protocol give the wallet access to information about the relying party? For example, could this wallet that I don't control tell its owner, or the government, that I am using it to access a certain website?
If you think you have a better idea shoot me an email.
> Controlled by users: The EU Digital Identity Wallets will enable people to choose and keep track of their identity, data and certificates which they share with third parties. Anything which is not necessary to share will not be shared.
I think where the ZKP stuff being discussed here fails to meet this criteria is the wallet provider is also a third (non-user) party. You stated elsewhere that a malicious wallet could leak data about a transaction: that's exactly the vulnerability that is not being accounted for by this protocol.
> If you think you have a better idea shoot me an email.
Sure, will do. It does seem to me like a solvable problem. I think this kind of tech is really important and I'd love to see this hole get closed so I can feel better about supporting it.
[1] See section 33 here https://www.european-digital-identity-regulation.com/Preambl...
Nothing stops a government from making their code open source and providing you with reproducible builds. You just won't be able to change the code to do something the government doesn't deem legal.
(2) One of the goals of this project was to layer ZK on top of current identity standards that DMVs already issue, so that gov orgs don't have to change what they currently do to support the strongest user privacy. One example format is called Mdoc.
(3) The user holds the identity information on their device only. No other copies. The user's device makes the zkp proof on-device. This was one of the major technical challenges.
(4) The relying party (eg a website) runs the zk verification algorithm on the proof that is produced by the device to ensure soundness.
(5) Yes, the user can use any compatible implementation to produce the proof. We have open-sourced our implementation and we have a spec for the proof format that others can also reimplement.
If so, what's the point of requiring your implementation to run on a verified secure element? If not, the protocol seems only as strong as the weakest chip, as obtaining just a single private key from a single chip would let you generate arbitrary proofs.
The ZKP library does not run on the secure element. It runs on the normal CPU and produces a proof that the ECDSA signature from the SE is valid (and that the ECDSA signature from the issuer is valid, and that the credential has not expired, and ...) If you crack the ZKP library, all you are doing is producing an incorrect proof that will not verify.
I tried to sketch out a design an age verification system, but it involved the DMV in each verification, which made timing attacks a problem. Briefly the website would issue a token, you'd get a blind signature of the token from the DMV's "this person is 18+" service, and return the token and unblinded signature to the website. I think that can be made to work but if the site and DMV cooperated they would likely be able to unmask many anonymous site users by comparing timing.
Getting the DMV out of the picture once your device is set up with the credential from them nicely eliminates that problem.
However, the timing attack resurfaces once you allow the DMV to revoke credentials. Exactly how the revocation is done matters. We are actively pushing back against solutions that require the DMV to be contacted to verify that the credential has not been revoked at presentation time, but this is a very nuanced discussion with inevitable tradeoffs between privacy and security.
> You are correct. The property that the colluding website and DMV still cannot identify you is called "unlinkability" and as far as I can tell cannot be achieved without zero-knowledge proofs.
Well, no. This is true only if you trust the unverifiable wallet software on your phone, which was provided by a for-profit, American big tech advertising company. In this protocol, the wallet may secretly leak the transaction details back to the DMV or whoever else they wish[1].
[1] "Yes, a malicious wallet could leak your information." https://news.ycombinator.com/item?id=44458549
Your observation that a bad wallet could compromise unlinkability is not a refutation of that. To refute it you need to show that it is possible to achieve unlinkability without using a ZKP.
By providing both attestation of individual attributes combined with "unlikability", how would even a single verifying party ensure that different attestations don't come from the same identity?
E.g. In the case of age attestation a single willing dissenting identity could set up a system to mint attestations for anyone without it being traceable back to them, right? Similar to how a single of-age person could purchase beer for all their under age friends (+ without any feat of repercussions.
In general there is a tradeoff between security and privacy, and different use cases will need to choose where they want to be on this spectrum. Our ZKP library at least makes the privacy end possible.
That seems a bit like a game of whack-a-mole where as long as the forging side is willing to go further and further into out-of-hardware emulation (e.g. prosthetic finger on a robot hand to trick fingerprint scanners), they are bound to win. Biometrics don't feel like they hold up much if you can have collusion without fear of accountability.
> Our ZKP library at least makes the privacy end possible.
Yes, that's also one of the main things that make me excited about it. I've been following the space for quite some time now, and I'm happy that it becomes more tractable for standard cryptographic primitives and thus a lot more use-cases.
Thanks for your contributions to the space and being so responsive in this thread!
What you don't get with this scheme (and which zero knowledge proofs can provide) is protection against correlation: if you sign into the same site twice or sign into different sites, can the site owners recognize that it is the same user? With the design of the core mdoc selector disclosure protocol, the answer is yes.
The form are eg things like the JSON Web Token (JWT), Digital Credentials, and the Federated Credential Management API (FedCM).[1][2][3][4][5] The software can be anything since they're expected to use open protocols, so yes, web browsers.[6] Per the Commission, "For remote presentation flows, … the Wallet Instance implements the OpenID for Verifiable Presentation protocol OpenID4VP in combination with the W3C Digital Credentials API."[7]
[1] https://en.wikipedia.org/wiki/JSON_Web_Token
[2] https://github.com/w3c-fedid/digital-credentials
[3] https://w3c-fedid.github.io/digital-credentials/
[4] https://github.com/w3c-fedid/FedCM
[5] https://w3c-fedid.github.io/FedCM/
[6] https://github.com/w3c-fedid/FedCM/blob/main/explorations/HO...
[7] https://eu-digital-identity-wallet.github.io/eudi-doc-archit...
Virtually everyone gets their internet from an ISP that is regulated in the country that the user lives in. There are no technical barriers to implementing a permitting system in the United States.
Linking connections to real people is self-enforcing when there is a usage-based tax.
[1] https://www.africanews.com/2018/04/13/uganda-s-social-media-...
Those remaining very likely have multiple advantages like advance technical knowledge, connection to powerful people in business / governments, money and legal support in case they end up on wrong side of law. There is very little benefit and lot of effort to catch these unless they are running some kind of criminal organization which adversely affects their government/regime.
People have bothered with downloading low-quality Mp3s from Napster, figuring out video codex and modding game consoles to get free video games. If the need is dire enough, the users will figure it out, no matter how high the friction is.
Those with enough technical chops will figure out how to do it by themselves, those with enough intelligence will find resources on the internet, the rest will ask a friend or pay a local IT person to get it set up for them.
That's not "enough", it was extremely nice and probably less than 1% of population
WG traffic is easily identifiable and able to be blocked, it's what happens in countries that ban VPNs.
I don't like this but don't have another solution other than the porn industry self-policing which isn't promising.
Somehow we've inappropriately shifted responsibility away from parents/guardians in some areas like internet access.
In other areas, like letting your kid go outside by themselves, we've criminalized reasonable caregiver actions.
It's a wild world.
And parents aren’t in control of children 24/7. Schools tend to provide tablets and laptops everywhere, and how much trust should parents have that things like a content filter are adequate to keep children from asking objectionable pornography, hate sites teaching misogyny and so forth?
I think most would agree that there's a significant difference between a physical product that shortens the lifespan of virtually all humans who use it, and looking at images and video, no matter how extreme.
> And parents aren’t in control of children 24/7. Schools tend to provide tablets and laptops everywhere, and how much trust should parents have that things like a content filter are adequate to keep children from asking objectionable pornography, hate sites teaching misogyny and so forth?
Agreed.
Parents and guardians should definitely be aware of and concerned about what internet filters are in place at schools.
Neither of the words you used give parents any control over the situation. Legislation is the circumspect way parents are exerting control over websites that are unable to police themselves.
Schools have traditionally been ground zero for culture war in the USA, so this fits.
I think the real issue is that the definition of "reasonable" is subjective and often changes with time/culture/people in charge at the moment.
For example, on Discord, all your messages are scanned for such. On Cloudflare as well (for over 5 years).
For now it means they have no interest to remove such content unless coerced or affected by the public opinion.
This would destroy all content though, not just for minors.
Absurd, but it works, in North Korea (death penalty), Iran (death penalty), China (10 year prison), and also protects victims from rape, or "rape" under financial pressure.
The alternative is to let responsibility of the parents to install web filter to their kids, and let others live freely on the internet, without sharing their history or IDing them.
In reality, TikTok also has really traumatizing content, yet is engaging tons of kids and teenagers, and IDing won't solve that, but good parents can.
I just pushed this idea as a "solution" to see what others think, but I don't know. Again perhaps educating the parents about how to educate kids about the dangers of internet, and perhaps a web filter for kids.
This is actually one place where AI could be useful, to do dynamic local content classification (instead of a blocklist), especially if integrated directly in Android / iPhone.
Like https://support.apple.com/en-us/105121 but more dynamic.
CP is an edge case but that's because it's almost impossible to make CP without abusing children and you could view CP as an incitement to violence -- as incitement to abuse children.
Parents should ultimately monitor what their kids do. I have a pi-hole that subscribes to lists with millions of porn domains, but I'm a technical person. Non-technical parents are helpless, and kids can easily access it at friends' houses etc. The industry has not empowered non-technical parents to do this, probably because there's a conflict of interest. Lots of parents would use such options to keep kids off social media, and like all addictive things social media wants to hook them early. (I think kids should be off social media too, but it's not quite as nuts as letting them watch fetish porn.)
Porn is different now too. It's worse in a way. Like everything else it's subjected to a pressure to get "edgier" to maximize engagement. So today's porn is loaded with simulated incest, simulated rape, extreme BDSM, etc., things that young children are not equipped to properly contextualize. (Some adults aren't either, but at least with adults you can say it's their fault not the porn's fault. The line cuts differently with children which is why children can't smoke, get tattoos, buy alcohol, get credit cards, etc.) If you want to see the consequence of young kids (mostly boys) being raised with unfettered porn access go visit any women-coded space on the Internet (like Reddit) and search for threads discussing why so many men want to choke their girlfriends. Where did this sudden choking fetish come from?
The "porn has been giving men violent sexual fantasies" line has existed since before I was born but it always ignores that they're the top fantasies among women too. Among my friend group the more common refrain is women who want to be choked but their boyfriends are uncomfortable doing it.
Perhaps find a way to force Windows / Android / iOS to include such "firewall"/webfilter by default.
I really like Andy Birrells "micro-cents" which exploited the fact you could not easily reverse an MD5 hash so you one could cheaply do high confidence low value transactions at speed. Another idea that never got anywhere sadly.
ZKP ID cards and ZKP currency are both interesting things from the 90's I'd love to see in real life. Imagine I could pay you phone to phone with no network level of capability using a currency that couldn't be double spent. That was the promise of digicash. The government hated it :-). It was just like cash currency in that serial numbers could let you track the bank it left, and the bank it came back in to, but you couldn't track anywhere it had been between those two points.
Fun times. I'll have to see if some of my ZKP ideas can be built on top of this tech now.
Do you still feel that way knowing that it introduces a hard requirement for all users to have their private data managed by one of Apple, Google, or Microsoft[1]? I want to be excited about this, and about Passkeys, but the people working in this space keep fumbling this ball :(
[1] "Using the MDOC requires a signature from a hardware security key in the phone" https://news.ycombinator.com/item?id=44458417
> To be very honest here, you risk having KeePassXC blocked by relying parties
Even if the bigtechs don't "officially" make the passkey standards require bigtech involvement, it seems very likely to me that conservative businesses like banks will only accept bigtech implementations. And then you're sunk.
Similarly, look at how OpenID turned into "Sign in with AppleGooFaceSoft".
This ZKP+hardware secure element stuff seems even worse, because how are you going to make it work on old hardware, or with free software, or with open devices?
Indeed. It's not a theoretical concern, either. The spec authors themselves actually maintain a "naughty client list": https://passkeys.dev/docs/reference/known-issues/
> This ZKP+hardware secure element stuff seems even worse, because how are you going to make it work on old hardware, or with free software, or with open devices?
I don't love it, but I actually do see an argument that this kind of proof-of-property stuff really does belong in a secure area, backed by approved software. It is making government-backed, legal claims about a person or entity. Unlike with Passkeys, it's not really "your" data, rather it's a way for the government to provide legally-backed information to someone, without the government actually having to be in the loop. I'd probably argue the solution to the big-tech dependency here is the government should be required to provide its own, verifiable solution (such as a physical ID card with open software) for users who do not want to trust big-tech.
Where the ZKP spec authors goofed was in not considering the wallet provider to be a party in the transaction. That third party may have interests that are not aligned with the user's.
If only the recipient doesn’t have access, a certain amount of trust can be delegated to the strength of the proof presented in the spend. In an ecash model, the proof would be in the form of a signature made by the mint (assuming the recipient was able to get the public keys the mint was using).
Active research is being done on the ecash model with the resurgence of the concept in the Cashu and Fedimint projects. Cashu takes the online sender, offline receiver approach[2].
[1] https://chaum.com/wp-content/uploads/2021/12/Untraceable_Ele...
^See paragraph in the introduction ending with:
“But if Alice reuses a coin, the bank can trace it to her account and can prove that she has used it twice.”
- Buys or borrows a laptop / phone / whatever from somebody with an authorized private key
- Downloads an authorized private key file from a sketchy forum (maybe hacked from an unwilling target, maybe willingly shared by a free-speech advocate)
- Uses a VPN over HTTPS to visit websites in countries where age checks aren't legally mandated (and non-compliance is implicitly or explicitly encouraged for economic or ideological reasons)
Just like with passkeys or MFA, the "something else" could be purely software though, right? And hence automated?
For example I can run Windows 11 in a virtual machine on Linux, using softu2f to emulate TPM 2.0, and Windows does not know the difference.
I should also remark that the above is a western-centric perspective, whatever "West" means. For example, I heard the architect for a similar system already deployed in India remark that in his jurisdiction many households share one phone across many family members, and India chose to accept more possibility for fraud in exchange for wider usability by the population. In that context this choice looks like the correct solution.
In the future, you'll need a signed certificate with your PII/KYC to access the internet and get an IP address. China is already on the way there and the west is warming up to this approach.
The reason I ask is that I know that many teams working in the b-word field are _regularly_ making great progress. So I'm just wondering if this work is actually novel / useful or whether it's Google releasing something that is already stale.
I'll just say that the b-systems solve a different problem, and for the problem solved by our system there is currently no other solution available.
We spoke with Ying Tong and her colleagues from the Ethereum foundation. They have a project investigating which ZK technology would be best for digital credentials, and they have ran a few benchmarks at https://hackmd.io/@clientsideproving/zkIDBenchmarks For reference, our implementation runs the benchmark in about 200ms on the same hardware. The ETHF folks have had access to our code for a while and they agree with this result, but they decided not to publish numbers until the Google code was open-sourced for all. Our system is thus about 10x faster than the closest contender for this problem.
I don't want to make any general claims about who is better than whom. Our system is designed for our problem, and it's not a surprise that another system designed for another problem would perform worse on our problem. We are big fans of the Binius system of Diamond and Posen at Irreducible, and there is a chance that Binius may eventually work better than our stuff. That's however not the case today.
You also have to be careful about which hardware to use. Our implementation is single-threaded no GPU because it has to run on all phones everywhere in the world. Whether or not one can do better on a high-end GPU is irrelevant to us.
Either way, "stale" is not a word I would use. The word I would use is "works today".
It's also an unfathomably complex solution [1] which only a few people in the world will grok, and far more complex than existing solutions such as Idemix or BBS+, which lack such a hardware binding on existing hardware.
Age verification in a privacy preserving way is a really hot topic at the moment, but it will always be possible to bypass it – as will any commonly held anonymous boolean – in quite trivial ways. For example by setting up an open proxy to disclose genuine attributes. There are some privacy preserving mitigations, for example cryptography that'll make you linkable when disclosing more than k times per time period, or detecting slower-than-near-light-speed disclosure in a face-to-face disclosure scenario.
However, these mitigations will never be completely secure. That might not be a problem if it's admitted beforehand so expectations are correctly set: it's a barrier to protect the naïve, not an impenetrable fortress. However, if the expectations are that only age verification that cannot be bypassed is "adequate", we only have to wait for the first incidents in production apps after which the open source and privacy story will be abandoned in the name of security.
[1] https://eprint.iacr.org/2024/2010.pdf and https://eprint.iacr.org/2022/1608.pdf
How are you going to check the document expiration date in BBS? Yes I know about range proofs, I know about the quaternion norms and the four prime theorem and all that jazz. But nobody is talking about it.
How are you going to bind to a hardware secure element that only uses NIST primes? Yes, there is a very clever variant called BBS# which I believe works, but that's not simple either.
How are you going to deal with existing standard formats? 80% of our complexity is in this step. BBS most likely cannot do it at all. If we can change the format then a lot of my complexity disappears too.
How are you going to deal with the fact that BBS signs an array and not a set, and thus you are leaking the fact that "family_name" is attribute at array index 42? Are you going to leak the schema (which re-introduces tracking) or are you going to agree in advance, now and forever, on a schema? (Our system hides the schema and works on an arbitrary key/value dictionary, up to a maximum size.)
It's easy to say "simple" when one has not built the real thing.
I disagree that no one is talking about it: the solutions are there, it is a question of getting the resources to put it together. Circuit based solutions have some nice properties, but the actual security assumptions are a bit odd, and the reasons people should trust a complex circuit and verification protocol are a bit hard.
I don't however think this is really the big debate. Rather it's about ensuring SD-JWT and related non-private solutions do not get used. To the extent that this work helps show it's possible, and the tradeoffs are desirable, it's good.
On that we all agree.
You can read the docs and whitepaper here: https://docs.reclaimprotocol.org/ And also take a look at all usecases built on top of this tech: https://reclaimprotocol.org/ecosystem
The providers are clutching their OLAP like pearls! :-)
Current benchmarks for proving costs are 33k txns per dollar and we expect this to go down x10-x100 over the coming months/years.
A system that can be trusted needs to work in the real world, with credit card payments, bank accounts, VAT.
I can also move the funds into a connected debit card so I can spend it in the real world.
This wallet is powered by Bridge.xyz which was acquired for $1B+ by Stripe in Fall 2024 - would encourage you to try the new stuff!
But how does it prove that the request is actually made by a person and not a bot? Surely that part is technically impossible right now?
How does the government guarantee that the natural person is such? Various jurisdictions will decide what's good enough, but as a strawman proposal, you go in person to city hall once and upload a document to your phone.
I know someone in germany that got detected cancer in an MRI scanner. The doctor gave him the images and told him to drive to a specialized hospital ~400km away. Otherwise they would send it there with a physical mail and the treatment would have started a week later.
The Sparkasse network is not very well known outside of Germany but is actually Europe's largest financial services group by assets.
What is interesting is that until the 90s the membership banks were public institutions backed by municipal and state guarantees that made them virtually bankruptcy-proof, unlike private banks. EU competition rules then forced Germany to phase out these state guarantees, making Sparkassen subject to normal banking regulations and deposit insurance like other banks.
Our team is leveraging zkVMs for paygo.wtf