I'm Joining CloudFlare
words.steveklabnik.com
words.steveklabnik.com
It's obviously not just Klabnik. In fact, when we did Starfighter a few years back, I think Patrick did one of these posts as well. But: like every other post about Starfighter on HN while we were working on it, the whole thing was pretty uncomfortable. It's an experience I'd prefer not to repeat.
There are lots of excellent people taking all sorts of jobs at all sorts of places, and, unless those jobs involve, I don't know, mechanizing payday loans to squeeze poor people or hacking the phones of people the government of Bahrain doesn't like, nobody needs to justify them. Some of these posts seem like they might be good examples of content actually better delivered in a tweet.
It's possible that I just don't think Cloud Flare has earned the spotlight Klabnik is giving here. Who knows. But I felt the same way when Yegge announced his job at Grab --- just, "why am I reading this?" I'm sometimes reminded of the way Anthony Bourdain described the trailing years of Mario Batali's tenure at Food TV (Batali was a "lion" but not in a good way). Like: did you want to write this, or did Cloud Flare reallllllly want you to write this?
If the former, people share all sorts of oddities in their own personal space especially life-changing events of which career moves is one.
If the latter, a lot of people follow "developer influencers" to keep up on the cutting edge trends. I'm personally keeping an eye on the Rust Book and hoping it maintains its high quality so that's my investment in this. I'd go the other way and say it would be borderline irresponsible of him not to publicly state he's moving off Mozilla's payroll b/c of so many of us depending on his super-active involvement in and documentation of Rust.
> why am I reading this?
Yeah, why? A better question is, why are you commenting on not wanting to read it instead of just skipping and moving on?
As to why I'm commenting on it: because it was submitted for our comment, and this is my comment.
Was it? Is there any particular expectation that HN users must comment on stories, as opposed to simply reading them and gaining whatever value they gain from that exercise?
I don't know about you, but I certainly don't feel compelled to comment on every HN post. And judging by how many posts hit the /newest page and disappear with 0 comments, I don't think I'm alone in that...
In the context of how it was used, that was absolutely the implication. Tptacek submitted that as the rationale to explain his commenting, when he (obviously) could have easily skipped the whole thread. Ultimately there was no real reason for him to bother commenting in the first place...
Then you come along and want to interpret that reply to actually mean 'Everything permitted on HN is compulsory'. That is, at a polite minimum, pointedly silly.
Me
> asks another why they are commenting which is one of the oldest, malignantly dimwitted things to say in an online forum.
Not following. There are many threads on the front page throughout the day. My usual MO is to read and comment on what I find interesting, not the converse. I find it strange to immerse myself in posts that make me "pretty uncomfortable". Like those people who tell me how much they dislike IPA's while continually sipping it.
> he other person, mindful of the site guidelines
I avoided mentioning it explicitly b/c tptacek is a regular contributor, but do some of his comments run afoul of "Please don't impute astroturfing or shillage"? Also "Why is this on HN?" is one of the oldest and least useful questions on HN since it's obvious things are voted up b/c some group found it interesting. If there's a serious problem with it, just use the flag. I expected better of him, honestly.
It's possible to write about how one is perplexed by what others find interesting, in an interesting way.
Also "Why is this on HN?"
It's not a 'why is this on HN' comment.
I expected better of him, honestly.
wat.
No, that is, at a polite minimum, your mistaken interpretation. And it has nothing whatsoever to do with what I actually said.
> Were people really record-scratching over his totally reasonable decision to take a good-paying job at a big company?
I actually anticipated a few people being confused because of a few reasons. It seems like, so far at least, nobody has actually been confused. It happens. But I've experienced a lot of that previously; so like, moving from "writing documentation" to "product manager" and "everything is MIT/Apache2.0 licensed" to "a closed source platform" can look like big changes from the outside.
And really, I have wanted to write this bit about edge compute for a while, this is just a good excuse to finally publish the piece. I've been talking to CloudFlare for a few months, and it felt slightly odd to write it before I told the world "hey I may have some bias here." Now that I've made my decision and that bias is clear, that helps further discussion.
Oh, one last bit I missed:
> Like: did you want to write this, or did Cloud Flare reallllllly want you to write this?
I brought up to them that I wanted to write a post about joining, and asked if they wanted marketing to look it over or something. They said "no need, we'll be happy with whatever you write."
I see it as a mutually beneficial thing, and always have, everywhere I've worked.
If you mean "Warp doesn't support generic clients", I commented about that over on the Reddit thread: https://old.reddit.com/r/programming/comments/b9sznr/im_join...
If you mean "CloudFlare pursued BoringTon outside of the general WireGuard project", well, I don't have any real insight into the specifics of why that was chosen, but it seems extremely open source to me. As far as I know, BoringTon is just an implementation of a protocol. There's a ton of reasons to want to do things on your own rather than work on upstream. And if anything, for protocols, independent implementations can be a good thing, more than a bad thing.
If you mean something else, well, I'm not sure, those are the only ones I can remember :p
Your previous rooting interest in this story would have been for Rust itself. And Rust itself would have been been best served by having it host the dominant sanctioned userland implementation of WireGuard, not merely as an implementation detail for a commercial offering that also treats WireGuard as yet another implementation detail. Rust? Important! WireGuard? Important! Warp? Who gives a fuck?
So this is a bit of a surprising response to see from you.
Apologies for the late edit:
I should lay my own cards on the table here.
First, it's not much of a secret that I'm no fan of big Flare. But just to be clear: good friends of mine have worked there, and several other people I deeply respect have as well, and I don't judge any of them for that.
Much more importantly: Jason Donenfeld is, to my mind, the hardest working person in show business, and he's set himself on one of the most important projects in network security, basically all on his own, and the momentum he's managed to achieve doing that is insane and a little inspiring --- especially given the amount of shade he's been repaid with by network security luminaries --- and basically the last thing in the world he needs is to burn cycles worrying what some giant corporation is doing with his work (which, sure, does not include this Rust implementation, but very obviously does include all the hard protocol design and validation work he's been doing over the last several years, which, if it were easy and straightforward to do, several people would have done before him).
I've done about as much serious technical work on WireGuard as you have --- to wit: none, though I am a loyal, diligent distributor of WireGuard stickers. My partners all feel the same way, so we've just been sending the project cash.
If your employer has done the same, and dumped a giant bucket of cash on Jason in exchange for having gotten so much mileage out of his work, that would be an excellent rejoinder to this particular set of concerns.
Eagerly, therefore, awaiting my comeuppance on this thread! :)
> then rolled it out as a proprietary offering that WireGuard itself is not allowed to connect to.
So, I said this in the link, but the implementation of the protocol itself is, as Zack said, fully open source. To me, that's the high order bit here. And as I said on the reddit thread, it appears from the outside that this is more of a support issue than anything else. Maybe that's a lie, but I'm inclined to take jgc at his word.
On working with upstream, what they said publicly is this: https://blog.cloudflare.com/boringtun-userspace-wireguard-ru...
> We communicated with Jason throughout the process and have a ton of respect for him and the entire WireGuard community. In the short term, we need the flexibility to quickly update BoringTun's code base to support the project we built it for. That's harder when you need to coordinate with people outside Cloudflare and when we need to move as fast as we plan to. However, we really believe in Open Source and want the WireGuard community to thrive. We licensed the code very openly (3-paragraph BSD) and WireGuard may choose to fork it. If they do, we'll support it and plan to contribute any improvements in our own fork back. Over the long term, I think we're very open to merging this back into the upstream project.
This seems very reasonable to me. YMMV. I have no idea what requirements would cause this friction at the moment, but peeking at https://www.wireguard.com/#contributing it looks like they do the "non-GitHub server + mailing list" for development. I mean, I'm literally wearing a GitHub hoodie, but I know that I personally prefer the GitHub workflow to the point where if doing work via that system were an upstream requirement, I'd probably do my own thing on GitHub too.
Basically, it may be good, it may be bad. I don't have enough information to really pass judgement.
:gif_of_homer_simpson_backing_slowly_into_the_hedge: is a totally legit response to this whole situation, for what it's worth.
(I'll reiterate: it would be amusing if your employer just smote me where I stand by revealing the large donation they've made to Jason's project).
My job isn't to be a PR person for everything Cloudflare does. Cloudflare has and does things I disagree with. It's a company with a lot of people working there, after all :)
All of this stuff is completely outside of what is going to be my job. And a job I haven't even started yet! I mean, sure, it's legitimate to talk about like anything is, but if you want to know what's going on, I'm certainly not the person to talk to about it.
It was a pretty weird move on Cloud Flare's part. I'm not saying it was "bad", just, weird.
I don't think any of us at Cloudflare had any idea Steve was writing this.
Similarly, no one at Cloudflare asked me to write https://sandstorm.io/news/2017-03-13-joining-cloudflare back when I joined, and I can't recall if I even told anyone I was writing it. That post also hit #1 on HN, which, honestly, was a huge surprise to me -- I didn't think it was that interesting, for similar reasons to what you say. I wrote it because it seemed like an obviously sensible thing to announce, but not really for attention-getting purposes...
I guess people are free to upvote what they think is interesting?
I technically told Billy, but I don't think he bothered to tell anyone else :)
Billy actually told Joaquin and I, but we were more psyched that Steve was joining than worried about how HN would find out.
Because many others on HN liked the story and upvoted it, it's not all about some individual's preferences.
"Intellectual curiosity" aside, it's also not all about discussing algorithms, startup strategies, and the nth JS framework this year. It's not like we don't get enough variety or that it's all "new job" posts...
The thing is once you do a number things in the open in geek-land, you become a short of mini celebrity in some circles. And people are interested to learn news about your career and progress -- e.g. of people like Steve, Katz, Abramov, Dahl, and so on.
Those sorts of posts also let people know how other people think about their career, the sort of process involved when interviewed/or asked to work with companies like Google, Cloudflare, etc (which someone reading HN and starting out with the whole programming thing from say Poland or Australia would not have the chance to know first-hand), and often what to expect from the person taking the job (e.g. I'll be working on X as part of my new job, and contribute back to the Open Source X).
Because he wrote it, and then people upvoted it, and then you clicked it. Same reason I read your comment. Why is it ok for you to comment on the article but it's not ok for him to write it?
Isn't most of this networking and self-promotion? It helps to announce if you have any kind of audience what you're doing so when the next opportunity comes long people will recognize who you are, what you've done? That's how I view all these posts.
It's the same when a startup posts a blog about a technology they've decided to use. You never hear a carpenter write a blog post about why he picked a specific type of hammer to do his job, but in SV/software/etc it seems pretty affordable (i.e. free) to kept yourself relevant in the eyes of strangers.
https://paulsellers.com/2013/10/debunking-myths-mysteries-sc...
https://freethoughtblogs.com/stderr/2019/01/29/a-guest-in-th...
(Yegge's rare recent posts are pretty transparently a hiring pitch.)
Congrats Steve! Excited to see how this turns out.
Thanks!
So I think the reason for "CloudFlare workers" not being available ten years ago has to be found somewhere else.
Specifically,
> But the kind of scalability I'm talking about here is scalability to the number of tenants, the number of applications that we can be hosting at one time. The challenge with this is that, again, we don't want people to choose one or two or five locations where their software runs; we want it to run everywhere. We want everyone's code running in every one of our locations. And some of our locations are not that big. Though some have lots and lots of computers, but others have maybe a dozen machines. How do you fit- we have 10 million customers- on a dozen machines? Turns out that the existing technologies for this, the existing server-side technologies, don't live up to the task. What we really need is basically 100X efficiency gain in number of people we can host and 100X decrease in how many resources each one is using.
- CF still has a less sophisticated targeting system than these larger companies that have been doing this for a long time. Targeting is everything when it comes to CDNs.
- Streaming games is totally different than serving web pages. It requires specialized hardware and targeting. It's basically a whole different business.
CF has a great strategy, but it seems to be focusing more on using their agility to adopt WA and Rust early and turn them into advantages.
Do you have pointers to public data comparing services?
Argo from CF is a year or so old and that sort of stuff was done like 10+ years ago at Akamai. Cotendo had a great system that was more impressive 5+ years ago.
Stackpath don’t have the sort of resources that CF has but have been doing edge compute for a while now (though not WASM).
Those are also not numbers I'd want to base a serious decision on since when I've benchmarked CDNs in the past it usually came down to things like reliability and something like 90th percentile numbers since the median numbers were usually fast enough but the outliers were what made people mad. Similarly, some providers had surprisingly high DNS latency (500+ms!) in various parts of the world but delivered competitive performance after that, requiring some judgement.
Lambda@Edge does run on their CDN nodes, but it isn't quite for general-purpose compute, there's a long list of restrictions you can find here: https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope...
If you're interested in the performance differences, I ran some comparisons using Catchpoint a while ago: https://blog.cloudflare.com/serverless-performance-compariso...
Lambda and Lambda@Edge also deal with things like cold-starts which impact performance irrespective of the breadth of the network. You can see some of those numbers here: https://serverless-benchmark.com/
EDIT: Fixed, thank you! I meant to double check this and just forgot before hitting publish :(
(Although since Rust can compile to WASM, it might be possible to write Rust, compile it to WASM, load that WASM in Node and run that on Lambda@Edge.)
> My role as part of Storage will be to consider “what does data access and storage look like in this world?” If your code is moved to the edge, but your data is still in a central server, you don’t gain the full benefit of having the code close to the client. There’s a lot of interesting stuff in this space!
Fixing this problem, or at least coming up with the plan to fix it, is now literally my job :) (There is of course already a plan in motion, given that Workers KV exists today...)
Yes, until we have actual edge storage, the use cases are more limited, but "edge functions" can be used for:
* Stateless / static use cases.
* Optimizing use of edge cache.
* Intelligently redirecting traffic to geographically-diverse back-ends -- including other cloud services that are already globally distributed.
For example on the last point: If you want to use Google Spanner as your database but you need to put business logic on top of it, then where does that business logic go? If it goes in a VM, either you have to make sure to replicate that VM all over the world (can be cumbersome and/or expensive) or you've lost Spanner's advantage of being globally distributed. You could put your logic at the edge, and then it will usually sit between the user and the nearest Spanner node to that user, which is ideal.
^F^f
Really looking forward to working with you!
And also note they're pre-IPO.
edit - Also there is a small spelling mistake in this line
> A lot of people see the realtionship
As part of this, you have Service Workers: https://developer.mozilla.org/en-US/docs/Web/API/Service_Wor...
> They are intended, among other things, to enable the creation of effective offline experiences, intercept network requests and take appropriate action based on whether the network is available, and update assets residing on the server.
So yeah, you can already do something like this today!
Wish you the best of luck!
It's always dangerous to predict the future, but one possible future for the web feels like it's starting to come into focus: javascript as the default language everywhere (frontend, backend, edge), plus the ability to compile lots and lots of other languages to WebAssembly.
A runtime that works everywhere (or, more accurately, a handful of partially compatible runtimes) will encourage moving processing around the network relatively seamlessly. Code will run on the backend sometimes, and in the browser and apps, sometimes, and in internet edge nodes, sometimes. This feels like the future we've been aiming for since we first added Java applet support to the browser.
There's a huge amount of toolchain and runtime work still to do to get there, of course. But it's a pretty cool future. Even (maybe especially) to language snobs like me.
The article describes the four eras of "how do I web server" as: physical servers -> infrastructure as a service -> platform as a service -> function as a service. Just to make that a little more anecdotal ...
I built my first websites in 1994. There really wasn't a commercial web, yet. To build a website in 1994 you probably either stuck some files in a special place under your home directory[0] (if you were at a university), or you downloaded and installed Apache to some machine of your own. Maybe an old machine you left running under your desk.
Then in 1995 Netscape went public and kicked off the "dot com" era, and Bill Gates wrote the famous "Internet Tidal Wave" memo. [1]
All of a sudden you could get paid to build websites! Big ones (or so we thought, back then). You probably needed redundant bandwidth, and a way to expand server capacity, and 24/7 power guarantees. There wasn't really a way to get that stuff except to buy hardware yourself and put it in a data center. You paid somebody with a fancy building a few thousand dollars a month (at a minimum) for a cabinet, and a committed "95th percentile" amount of bandwidth, a power connection rated to a certain number of apps, and a network drop. Then you bought a lot of machines from Dell or Sun (depending on your budget), spent a lot of time setting them up, and drove with them in the trunk of your car to the data center.
Then the first generation of "infrastructure as a service" (Rackspace) made the logistics a lot easier, and the costs a little better. Then AWS made the costs a lot better. And now "platform as a service" offerings like Elastic Beanstalk have made the logistics even easier still.
But, arguably, the as-a-service evolutions haven't made it possible to actually do new things I couldn't do on my own hardware. Faster, easier, cheaper, yes, definitely. And those are really important!
But wasm-at-the-edge feels new and, maybe, transformative.
I really like this paragraph about "the future" at the end of the Cloudflare announcement about supporting WebAssembly [2]:
"We're excited by the possibilities that WebAssembly opens up. Perhaps, by integrating with Cloudflare Spectrum, we could allow existing C/C++ server code to handle arbitrary TCP and UDP protocols on the edge, like a sort of massively-distributed inetd. Perhaps game servers could reduce latency by running on Cloudflare, as close to the player as possible. Maybe, with the help of some GPUs and OpenGL bindings, you could do 3D rendering and real-time streaming directly from the edge."
[0]-https://httpd.apache.org/docs/2.4/howto/public_html.html [1]-https://www.wired.com/2010/05/0526bill-gates-internet-memo/ [2]-https://blog.cloudflare.com/webassembly-on-cloudflare-worker...
WASM's purpose is to compile languages to run in a web browser. A CloudFlare "worker" is a web server that runs at a data center close to the user.
For another, I've been posting here for ten years. https://news.ycombinator.com/leaders has me at #19. (I'm a fan of patio11's "this is basically an odometer" take on that, of course.) My "I'm leaving Mozilla" post got a lot of attention too, as well as most of my previous job change posts. I know I like hearing about what HN members are up to, and I guess other people do too.
And your post was technically interesting too. I hadn't caught wind of WASI or WASM being run on edge services, so that's given me something to read more about.
Must be a fun place to work at. Is Cloudflare tracking this post?
Since the moderator was me I can tell you with high confidence that the moderator's decision was based on superficial pattern matching and did not involve any evaluation of corporate comfortableness. Also the idea of doing that nauseates the moderator at least as much as it does you.
I've read through every comment on this page and there's nothing here I wouldn't stand behind, even if much of seems to be misunderstood as in this thread.
Rust is also hedging its bets on WA, and it's looking like Rust might be the language of the future web on the server and client (thanks for all the fish Javascript). CF investing in Rust is mutually beneficial. Rust is still new, agile, and will let people write safe code, which is going to be critical when running modules in the end user's browser.
Hope they let Steve spend a lot of his time continuing to make Rust great, and hopefully Steve will encourage the other Rustacians at CF to make Rust better for the rest of us.