Nghttp3 1.0.0 – HTTP/3 library written in C
nghttp2.org
nghttp2.org
Other nice HTTP/3 libraries include H2O: https://h2o.examp1e.net as well as Facebook/Meta ProxyGen: https://github.com/facebook/proxygen
But H20 hasn't been updated for years. 2.3 Beta 2 was released in 2019.
Also see: "switch to a new release model" https://github.com/h2o/h2o/issues/3230
"Every commit to the master branch is considered ready for production"
H2O is absolutely battle proof -- its h2 and h3 implementation perform amazingly well.
I'm using Apache at the moment and it's working great. HTTP/3 isn't even in the horizon for Apache, and it is the reason why I'm trying out Caddy/Nginx/OpenLitepseed in the first place.
Would nghttp3 eventually bring HTTP/3 to Apache?
You know, Apache is pretty decent.
I'm actually surprised that they added ACME functionality with mod_md before Nginx, so now you can get Let's Encrypt certs without the need for certbot: https://httpd.apache.org/docs/2.4/mod/mod_md.html
If you turn off .htaccess the performance is okay as well (closer to how Nginx has nginx.conf), their mod_security performs pretty well and I'm not sure of that many alternatives to mod_auth_openidc elsewhere.
Of course, Nginx, Caddy and other web servers are really nice too!
None of the Nginx builds available out there seemed to fit my use cases perfectly, so I resolved to customize it (https://gist.github.com/Ayesh/7c7a1bf08d37497aef8672644a7fd5...) to my exact use cases.
I gave up on Nginx because it was impossible, or incredibly difficult to figure out how to append/modify HTTP headers. I wanted to send CSP headers if the reverse proxied application did not set on their own, and I wasted a whole fruitless day trying to do it. I gave up.
https://en.wikipedia.org/wiki/Nginx
Nginx (pronounced "engine x"...
There is no comparison with C, where trivial programs exhibit UB and memory unsafety all the time, and the standard library is covered in foot guns.
I really dislike the puritan wing of the rust community who see the unsafe keyword as some sort of blight to be excised. Unsafe is a necessary and important part of the language. It is needed for lots of reasons - like how you can’t make high performance container types in purely safe rust. Or to interact with code written in other languages.
Unsafe is not a devils mark. It’s a good and useful tool. Get over it.
Memory safe would be 0% unsafe.
If reliability is what you’re after, that time is almost certainly better spent adding more tests, fuzz testing and validating your code with Miri and friends.
Why are these tools written in C? Because they're old. nghttp2 was released in 2009, before Go was released and before Rust was even a twinkle in its eventual creators' eyes. Adding comments like this any time a feature announcement to a C library is made comes across like you're saying nobody should ever update tools that are already in use. Should Linux just stop doing work until the kernel is rewritten from the ground up in Rust? Should all longstanding software take a five-year pause to do total from-scratch rewrites? Can you see why that is impractical? If you can, then you can see why libraries like nghttp2 exist and continue to add features.
No need to rewrite everything from the ground up: https://github.com/cloudflare/quiche#curl
Simple solution - don't
nghttp3 can be used to create qpack standalone binary which does not touch the network.
https://raw.githubusercontent.com/ngtcp2/nghttp3/main/exampl...
Dismissing a new feature in an existing library seems quite excessive and possibly zealous. Or perhaps we should just stop using OSes entirely because they continue to release new features in memory unsafe languages.
Rust? why would want to write SAFE software in a language without formal spec? I question the morality af those people.
And it's because most security vulnerabilities these days have to do with memory issues, which must be manually enforced in languages like C. In Rust you get most of that for free, often even more optimized than being hand-written.
Also, in my opinion, Rust is also just a nicer language to use than C - as much as I love C, I'm wildly more productive in Rust than in C/C++, and have a lot of peace of mind knowing I can more strenuously audit the behavior, security, and other aspects of the code with relative ease.
I suggest trying Rust, understanding it, and then making a judgement call. Perhaps the community is 'loud' about it for good reason.
also, in my and industry with much more as the stake, opinion, Ada is better.
i suggest you to try Ada.
A formal spec just means someone took the time and effort to write one down.
A formal declaration that a language makes no attempt at preventing memory safety is not safer than an informal declaration that a language is memory safe.
For me, it translates to "a piece of software universally compatible with a vast amount of compilers and platforms that is probably trivial to compile even in 10 years".
https://www.cisa.gov/news-events/news/urgent-need-memory-saf...
I imagine their emphasis is on the new bit here.
Just because we sometimes have to make compromises for pragmatic reasons doesn’t invalidate the original point.
Not really. We just don't know the failure modes of the new hotness, and that makes us feel safer somehow. (Lol)
Your point stands though.
Working title: Semantic Paint.
---
The plan is to take what our web does poorly and do those things well. From that I have two main goals, the first of which is:
Permissionless Annotation -- I should be able to attach annotations to datasets (or subsets thereof) that I find in the wild without having write access to those datasets. Links are implemented as annotations (as are edits). Unlike our web, they are undirected and might link more than two things. Instead of a directed graph between documents we now have simplices which connect (sub)sequences of arbitrary bytes. These connections are typed (I'm calling these types "colors").
Have you ever played Mad Libs? It's a game which has partial sentences, like: "____ had a great time ____ing the ______". Fun is had by filling in the words before you know the sentence and then laughing about how silly the sentence is. In semantic paint, colors are like that: they're tuples with an associated partial sentence, each tuple element goes with a blank. At any one moment, your client will be configured to display (or act on) one or more "colors". A color is a list of tuples in this form.
So you might have a 3-color:
______ (code) is malicious, writing it to stdin of ______ (executable code) with ______ (parameters) will write a non-malicious copy to stdout.
This color would be used for annotating malicious javascript with enough metadata to fix it automatically. It functions as a link between three items. If you have any one of them, you can find the other two.
Here's the browser-killing part: at some point we stop annotating the malicious parts so that we can cleanse them, and instead we target the desirable parts so that we can make them more accessible. Embrace, enhance, extinguish.
Note that we're not talking about the filename or the server whence the malicious script came, we're talking about the data itself. Naming things is hard, so I want to see how far we can get without naming them at all. Instead, a user can just just point at the thing without naming it, and apply paint---er apply annotations--to the thing they're pointing at. It operates on fragments of data scraped from a screen, tee'd from a pipe, or OCR'd from a camera--not on files or other named abstractions.
The tuple values are pairs: a cryptographic hash, and a list of features that come out of a rolling hash (think rsync). The later is used to re-anchor the tuple (brushstroke) even if the canvas is paginated differently or has other small differences. Fingers crossed: I can keep false positives down to a tolerable level.
For instance, if you copy some code from stackoverflow into your project, and later I annotate that code while browsing stackoverflow, you should then see my annotations on your code as viewed in your IDE (that is, provided you have opted in to seeing my paint, and are running your IDE through a semantic paint client--software which will sit between you and the IDE. The first draft is shaping up to resemble tmux).
One could imagine similar functionality on a piece of paper you found blowing in the wind. Point your camera at it, extract the features, query... maybe there are annotations on that text which will tell you more about its origins. If Fermat had had this tech, he wouldn't have complained about the margin being too small, he'd just have linked the proof with a brushstroke.
Imagine also people with allergies leaving annotations on menus at restaurants: "they say this doesn't have gluten, but it totally does," that sort of thing.
In this sense you can think of it as a sort of distributed search algorithm, where either a cryptographic hash of content, or this list of fuzzy-hash features, is the search query. You'd just sort of leave it running as a filter over whatever data you're working with. I kind of imagine it like augmented reality... for data.
---
The second thing I want do do well is Partition Tolerance.
I want apps using this protocol to function without a persistent internet connection. They'll function slowly, but how up-to-date do you really need that blog post to be anyway? For most things, days or even weeks of latency is ok.
If you're in your car, stopped at a light, your device will be gossiping with others that are stopped at the same stoplight. Pedestrians in range may also end up participating. Delivery drivers put nodes on their vans, which silently gossip brushstrokes while the drivers deliver packages. Imagine a train full of people with gossiping devices... Sneakernet, on autopilot, even in a disaster or a protest.
Secure Scuttlebutt Protocol comes to mind here, but that's append-only. This is unordered. You just grab all of the brushstrokes you're interested in and provide strokes that your peers are interested in. Retention policies and algorithms for deciding what "interested in" means, are the domain of the app. Convergence will be hard to orchestrate, but that's no reason not to try (or maybe instead we diverge).
Peers come and go, but since everything is content addressed (cryptographically, or fuzzily), what really matters is whether those peers are interested in the same colors that you are. I know it sounds crazy ambitious, but if you don't have to protect the referential integrity of a globally consistent name, lots of problems go away.
The goal is to keep data nearest the things in the real world that it is relevant to. If you run across contradictory entries within a color, you can scrutinize by author (who do you trust more?) or by which peers gossiped it (which is more local?). I anticipate that handling trust explicitly like this (and focusing on data, not server names) will change the game re: misinformation.
One thing I like about this strategy is that you can synchronize this gossip (like cicadas synchronize their mating habits). I want to be ad-hoc wifi/bluetooth tolerant, but now imagine a node running in the cloud. Rather than leaving it on 24/7 so that you're ready to respond to a user at any moment, you can have your node on a 5 minute cycle: Sleep for 4:30, gossip for 0:30, repeat. That's paying ten cents instead of a dollar for server uptime. Yeah, users will have to tolerate data that's 5 minutes stale, but for most applications that's fine. If the data is relevant to them, it should make it to their node before they go to look for it.
Another benefit is that you can enlist your node to someone else's cause without having to talk with them first. Much like how IPFS lets you "pin" data published by somebody else so that that data doesn't go away if their node goes offline, you could instruct your node to notice square pegs and square holes and publish annotations about having fitted the peg in the hole. This means that if you don't want to pay somebody in money, you can instead pay them in operational support:
> I don't want to pay you $5 / mo. Instead I've been hosting way more than my share of your service on this stack of hard disks for a month. Let that be my payment.
I think there are some things that capitalism and zero-sum games are doing poorly, that cooperation and reciprocity could do well. That idea is not fundamental to the protocol, but it is fundamental to why I want to build the protocol.
----
Whew, I could go for pages and pages, but that's my best shot at a sketch. Thanks for asking, sorry it's not elevator-pitch-grade.
The file/folder metaphor has had fantastic success but it leaves people constrained to hierarchical thinking when databases can actually represent arbitrary graphs, cycles and all. When prototyping my knowledge graph I wanted to build directly on existing filesystems because readdir is fast and cached by the OS, a lot of work done already. But I was stymied by the fact you can't hardlink a directory to multiple directories, because cycles aren't allowed, so I'm stuck with softlinks.
I also weighed the pros and cons of tuples (unnamed associations) and triples (subject verb object, verb being the name of the association) and decided there's utility in having both without too much added conplexity - tuples are just triples with NULL for a verb.
Let me ask you this - architecturally, I'm still weiging building on top of unix filesystems, with git for history/multi-user collab sync VS sqlite and figuring out history/sync later, but take the abstract case of tables with columns: Does it make more sense to have only two tables - one for associating hashes/uuids with strings/buffers and the second table being "left id, right id" for links, or is it worth the added complexity to create a new table for each type - a specified set of named attributes each with specified type, such that when you want to pull up all the metadata on an mp3 file you already know which table to read, instead of doing dozens of reads on the simpler model and then verifying the type ex post facto... would love to know your thoughts on how to structure the graph on disk...
fwiw the tagline for my project is "global media graph of associations, attributes, & annotations"
Aside:
I think it's a tragedy that bit torrent is associated with piracy because it's just a better way to move data around in general. In order to not have that happen to me, I'm seeking a first application that is unlikely to ruffle feathers, so I've been taking a bioinformatics class. As I get familiar with workflows in the genomics/proteomics world, I'm finding that this:
> The file/folder metaphor has had fantastic success but it leaves people constrained to hierarchical thinking
Is resoundingly true. We end up with filenames like: "ncrassa.H3K9me3.ChIPseq.subtracted.merged.bed" which is not a name so much as a directed graph, a recipe of how this file came to be. Coming up with names for the resultant files at each stage in the graph and keeping them straight is this chore which could be dispensed with if what you were looking at was the graph that we're all holding in our minds while we write these names.
Semantics Note:
> tuples are just triples with NULL for a verb.
The way I think of it, pairs are just triples with null for a verb. When I say tuples I mean things that can have arbitrarily many values. Anyhow, I think I know what you mean.
As for data representation, I don't really like triples in the subject, verb, object sense. It's very Semantic Web™, particularly it's the vision of the semantic web which Doctorow argues has failed here: https://people.well.com/user/doctorow/metacrap.htm
I agree with his critiques, and I would add this one:
> People struggle enough to do their jobs as it is, if yours is a system that asks them to do something more in the name of useful metadata, you will be ignored.
If you have write access to important data, you have way too much responsibility to way too many people. We need to split that responsibility up so that different personae can handle different things. If I'm the one that wishes that this data had annotations, let me be the one to annotate it, and don't make me seek approval first.
A consequence of this is that you now need an "according to whom" field on all of your triples. Maybe you don't trust that particular author. Fine, quadruples then? Maybe, but when I actually try to get a complex problem to lay out nicely in this way I end up with a sort of predicate soup--which is how I ended up with my mad libs approach: arbitrary n-tuples. In some sense I'm passing the buck and making it the app developer's problem (which is, of course, also me, but wearing a different hat).
I'm not enamored of reddit these days, but when it was new it was quite innovative to just let any-old user create any-old subreddit. Most of them fizzled out into obscurity, but occasionally a community had its shit together, and those thrived. That's a dynamic that I want to lean into. If the data sucks, don't find a different protocol, find a different community of data curators using the same protocol.
So that relates to your question because unless you're going to be in-the-loop as arbitrator of everything, people are going to disagree, and there will be a certain amount of churn as they align themselves with different sides of that disagreement.
I have to imagine that at some point somebody is going to want to express something new. To borrow from my world, a mad lib:
> ___ and ___ played in superbowl ____ and ____ won (according to ____)
I'm pretty sure that this quintuple can be normalized down into some set of triples which sort of reference each other when the proposition needs to be reconstructed, but when you decompose it like that you lose the feeling that it's a single thing--a thing that can be trusted or not by a user--a thing that can be gossiped between users--a thing that can be purged once it's no longer needed. It may be the case that having to reconstruct the more complex thing while performing these operations is actually more complex than just paying the up-front cost of having them be more complex things in the first place.
So I guess I have to ask: Is it definitely more complex to have a separate table for each type? Or does that complexity just bite you at different times?
I once had to test assertions about a rather large database over a rather slow connection. I was very fortunate that I could strip off the foreign keys and just flat-out ignore tables that were not related to my testing (this required that I sync much less data). That's harder to do if the "real" things are expressed in terms of simpler things which now have to be teased apart.
My opinion is relevant to the "sqlite" side of your dilemma, I think. I'm personally a big fan of git and filesystems--they are among the most potent tools that I feel comfortable using. I also considered wrapping git to handle conflict resolution. But ultimately it's the road I didn't take, so I can't really weigh in there.
I avoided it for.... weird reasons:
Generally, a git repo is run by somebody with authority. They accept or reject PRs in a rather top-down way. That feels like the "consistency" side of the CAP theorem, and I have chosen "partition tolerance" instead. It's an idea I got from Unison (https://www.unison-lang.org/). If you don't do globally unique names (such as the URL for a git repo) you don't get fights over where the authoritative name points. Instead, each fork is created with equal validity and its up to the users to decide which one to install.
It's kind of like back in the day when BTC forked. The optics of the scenario was that segregated witness won, and the losers of that fight went off and created BCH. But realistically it was was just that there was first one protocol, and then there was a choice between two new protocols. The notion that one side of the conflict got to carry the old name is well... not as content-addressed as I'd have liked.
I have some designs for a Plan9 kind of thing on top of ssh/git/torrent
Similar to the Holochain project, which is a distributed datastore with application layer and sums itself up on the homepage as "Think BitTorrent + Git + Cryptographic Signatures" but I don't enjoy holochain. It's one of the attempts to rewrite the stack tho, see also Urbit.
I'll check out holochain, if just for contrast. I'm a bit leery about "chain" though, blockchains go to great lengths to achieve globally consistent outcomes, and I don't think people need that for most things. Even about money. Especially about money. If you're rich in here-dollars, and you go there, you shouldn't expect them to respect your here-dollars unless you earned them doing something that is also appreciated there. Liquidity comes from shared values, and different people often have different values, so like... drop the consistency criterion.
I know a few people with similar projects. We had put so much hope into what we thought the web could be, and we're pretty damn disappointed with what it is, so you've got a generation of nerds trying to build what we thought it could be.
Holochain is unfortunately named since its about as much of a blockchain as git is (merkle tree, no proof of work mechanism) but they obscured it willingly by trying to integrate ETH [0] as a way to pay peers for cloud capacity / capitalize on the crypto hype.
edit: "Liquidity comes from shared values" this is an interesting axiom, I'll have to think about that. I've always held that the neat thing about money is it allows collaboration between groups with opposing values, because at least we have one motive in common
> money ... allows collaboration between groups with opposing values, because at least we have one motive in common
...only really holds if our common motive is that we both don't want to be killed by some Leviathan type thing (originally this a Roman soldier who would kill you if you didn't trade Caesar-face coinage above its buillion value). Money today is just a proxy for the same old violence as before.
This places a cap on how legitimate money can be. The have-nots, should they ever manage to coordinate, would be better off turning their backs on it. When a rich nation shows up and starts throwing money around, you can be sure that the poorer nation is getting screwed. Probably there's some kind of violence hanging around ensuring that they don't act up. See the history of US policy in South America in the 70's for more on this sort of thing.
If we want to move past all of this yucky stuff I'm talking about, we need to your acceptance of my money synonymous with me proving to you that I've done something that benefitted you specifically. As it is now, it's just proof that my activities somehow benefit a bank somewhere.
Personally I think either QUIC makes it into kernels, or it will have a loooong time ahead of it with language- and vectorized/batched IO in the OS (maybe even down to the NICs?) catching up. Even the more mature implementations struggle compared with TCP today, for things like high bandwidth on consumer hardware. Not to mention CPU overhead and the battery drain that comes with it. (At least from my own high-bandwidth experiments)
Yes, I know a large part of web is already http3. But remember that http is used outside of browsers and data centers. I don’t know enough to back any specific proposals, but to me it sounds a lot easier to fix the tcp handshake, open 2-3 conns for the HOL-blocking issue, than to rearchitect the entire stack (and add new features) under UDP. I’m saying this as someone who is still very bullish and excited about QUIC.
Yes, but you have to frame the exchange correctly or people will look at you strangely when you start addressing and acknowledging them.
It's not like implementing transport encryption and multiplexing and congestion control and etc etc etc. I doubt many people in the world actually know how to implement QUIC. I implemented tcp/ip as a teenager when I was bored over summer break.
Given the lack of any function names addressing TLS/CAs/encryption/etc I'm going to assume it's like all the other HTTP/3 implementations and fails to fully implement the spec. It's funny. Everyone defends MS/Google's HTTP/3 based on QUIC by saying the lack of ability to use plain text or self signed certs is the fault of the HTTP/3 implementations. But all the HTTP/3 lib implementations say it's the fault of QUIC.
Not only in its public API, but those are missing from its source code as well. I can't see where it could be doing any encryption—it doesn't even link to e.g. OpenSSL.
> nghttp3 is a thin HTTP/3 layer over an underlying QUIC stack.
Really? Does trusting a self-signed cert as a root cert not work in most libraries? quiche at least lets you control the parameters of their TLS implementation (https://docs.quic.tech/quiche/struct.Config.html#method.with...), which lets you go as far as allowing you to manually verify certs yourself (https://docs.rs/boring/latest/boring/ssl/struct.SslContextBu...).
Also quiche supports self-signed certificates (I use it with them). In fact I'm not aware of any raw QUIC or HTTP/3 implementations that don't allow implementing your own certificate handling.
There's absolutely no way I can get my self-made root cert in the trust store of the major browsers so that a random person I've never met can connect. HTTP/3 is designed for the use cases of non-human persons. For human persons it's a terrible protocol.