How browsers really load web pages [video] | Hacker News Reader

How browsers really load web pages [video]

fosdem.org

191 pointstodsacerdoti74 comments

Biggest takeaway for me is just how terrible node.js is as a server. Can't say I'm personally surprised, but I think people need to wake up to the mess that moving to self-hosting in javascript has brought. There's no reason to use js outside the browser, there are so many languages that are better suited to build processes and serving content. Testing is different, but why not just run a separate js interpreter specifically for that?

I understand if js is the only language you know, it's tempting to reach for it. But I'm currently actively avoiding js in the toolchain of all my web projects, and my sanity is slowly returning.

Feeling absolutely the same way these days (with 10+ years of experience with JS). My current breath of fresh air is Go and Swift (despite the fact XCode is the worst IDE I have ever worked with before). Curious what’s bringing back sanity for you?

I'm using bash for simple build problems like building dynamic component lists, when bash gets too unwieldy I switch to python. If the build process is stable enough and the scripts are beginning to get long, I'll rewrite the script as a CLI tool in a typed and compiled language.

Rinse and repeat.

The only thing that's really missing to me is bundling, which feels like a bit too big of a project to tackle on my own. But my current projects aren't big enough to need that, I'll pass that hurdle when it comes. And I may have to run webpack as a separate build process through node, but luckily I can do that in a CI container, and keep my dev machine clean.

Wait, what's wrong with it? For all the stuff I've written in Typescript it's been great. And the event loop of JS performs very well as well.

> the event loop of JS performs very well as well

that's the very opposite of my experience! Or rather it does perform, in the sense that I don't think there's much margin for improvement, but the very concept of single-threaded server that switches context on IO is so flawed and painful.

    - Any vaguely CPU-intensive operation is a risk for latency to shoot up across every request.
    - You can never trust your monitoring: if I see that latency for an endpoint spiked, is it because something did happen to this endpoint OR an unrelated request hogged the CPU OR because the metric collection itself was CPU-starved
    - CPU profiling of a request is near-impossible because all requests are handling in a single thread
    - It doesn't play nicely _at all_ with k8s CPU resource management

I spent many hours looking at various dashboards, trying to guess whether what I was looking at was a cause or a symptom or just noise. Any hypothesis of what is going wrong needs to take into account the weirdness and side-effects of the event loop. As much as I love the Typescript language, operating Node is a nightmare.

I saw the same issues with my servers.

I'm thinking about porting to Bun, as it's web server module is implemented in Zig.

Could have the benefit of using TypeScript without the issues of Node.js' HTTP module.

Also, in case you haven't guessed it yet, all of this can be used to fingerprint a browser's identity--but not a user. In my previous company we had a piece of software that reported all sort of HTTP/2 packet traces, which then we fed to a machine-learning algorithm to know which connections were actual browsers and which ones were bots[^1]. It worked fairly well, but we never had to flex it very hard because it was at a time when most Internet bots were still running in HTTP/1.1 while most actual browsers were running HTTP/2.

A corollary of the above is that image-identification captchas in mainstream CDNs are not what they purport to be, but something darker and weirder.

[^1]: bots not powered by browser automation, that is.

I am grateful that he decided to mask himself as clown during this very informative presentation, it helps my sanity in check.

Especially after debugging performance issues on badly maintained nextjs app and not knowing most of this things.

Would be interesting to see if Bun or Deno perform better here.

he hasn't merged a PR in 3 years but https://github.com/alex/what-happens-when

Oohh I haven't seen this, will check it out as a possible metric / sample answer for this question!

1,800+ Para, cripes.

What is going on with Alex?

*PRs, sorry for the typo!

It's cool people are so interested in enhancing the document with creative new angles. Also too bad / a bit sad nobody is triaging the pull-requests.

too outdated. insufficiently complete even 3y ago.

someone tries to update the info with an updated PR almost every day.

One of my favorite tech interview questions for any position is, "Describe in excruciating detail, everything that happens when you request and load a website in your browser."

You learn a LOT about how a candidate thinks and the depth of their knowledge.

i feel this question correlates more with the generation of the SWE more than anything.

expecting current gen SWEs to talk about network layer protocols while answering this is kinda the same as expecting 1990s SWEs to include wire physics and dispersion statistics in their answer to this question.

Depth alone isn't always a good indicator. We have to move on from some of the low level stuff at some point and it is okay for engineers to know in detail about things that have been solved long back.

That's the beauty. I could care less if my backend JS candidate knows anything about Ethernet frames, but I damn well expect h3 and async discussion.

But my devops guy? He better be talking about CDN, cavhing, WAF.

Do you prompt them? Expecting them to talk about CDN and WAF seems unfair, those go beyond the definition of the web.

For a senior role? no. They should be asking you how deep you want to go. And you should be the one stopping them for time. If your SRE/DevOps can't wax poetic about the deep deep details, how are they going to get you out of the ditch when production goes down at 11am on a Tuesday, or 11pm on a Saturday

I’d expect an SRE to know about all that.

> CDN and WAF seems unfair, those go beyond the definition of the web

Current web is full of WAFs and CDNs, devops should know that they exist at the very least.

what is h3?

HTTP/3

Surely that depends what your hiring them for?

The nice thing about the website load question is that it touches every part of the stack. You could talk for an hour about rendering on screen at the OS level, or network protocols, or server stuff, or web client stuff, or data center stuff, or …

Really to answer the question in its full entirety would be the equivalent of that “Everything that goes into a pencil” essay. You could build an entire college curriculum just out of that question.

> that “Everything that goes into a pencil” essay

which one! I tried googling this but without quotes it gives me clickbait and with quotes it only points to this comment. :(

I, Pencil

thank you!!

100% a generation thing.

Been around long enough so I've written my own CSS framework, server-side framework, web browser, web server, sent bits over Ethernet, written assembler, programmed a FPGA, built circuits, AND have a electrical engineering degree... and YET, _ABSOLUTELY NONE_ of this is useful _99%_ of the time.

So meh. If I want you to do frontend, I will ask you frontend questions. Hopefully you can go deep on a11y and that's what I care about.

(But the 1% of the time when I can precisely step through a whole stack is also fun.)

Exactly, someone who can answer where media queries fall in terms of loading priorities for FE (when hiring for that.)

Electrical engineering is a whole nother thing!

Hence the differences in the level of schooling. Graduating a coding boot camp for using React vs years of engineering school. Why are the interviewers are confused on this is the unsane thing.

Yes, I would drop the “in excruciating detail” and “everything”. The level of detail the interviewee starts at or goes down to by default is also informative.

Why should "current generation" SWEs know about network protocols? We live in a more networked world, not less.

Oops! Bad, meaning-reversing omission: I mean "not know".

I think it's fine to ask a web developer how web pages are served.

If they can talk about network layer protocols, then that tells you something. If the next candidate's understanding stops at knitting libraries together, then that's also notable.

Even if it's not a discriminating factor in hiring, it still helps you flesh out the candidate.

I assume we have to start with the physical action of typing in a url and everything involved in that? Then how the physical keyboard works? Then how the OS handles the input from the keyboard? Then…

That's the beauty... Sometimes too much detail can be distracting, so knowing what to mention/omit is part of it too.

As CTO for multiple companies and hiring across multiple competencies for 10 years... This is the question that keeps on giving!

> too much detail can be distracting

I don't get it. You ask for excruciating detail. But you want candidates to intuit what to leave out? Eh?

You guide them a little. No point spending most of an hour describing keyboard technology.

Consider, do you type or paste the url? With or wothout http/s? Or www? Does it matter? What if you typo wwww.domain.com?

And what do you mean physical keyboard, the virtual one on my phone works fine ;) (also preconceived notions)

Which branch, physical or virtual. How does the virtual one work?

For a relatively in-depth take on this question: https://github.com/alex/what-happens-when

too outdated. even at the time of last commit (3y ago), lots is missing.

I mean yeah, it doesn't even touch what is happening inside your body in order to actually press those buttons. Or what forces are being converted and how they affect the surroundings.

And entropy when your browser garbage collects and free()

Or the decision to do it, the neurons involved, free will or otherwise.

Basically to load a web page you must first invent the universe.

Went for an interview, ended up with the existential dread.

I think it's funny to imagine the existential dread as a single unique instance of emotion that everything else feeds back into. Like - no, I'm not "having existential dread again", it's the same existential dread I've been having since the beginning...

This would be an interesting way to frame an entire series of textbooks covering pretty much all of science, electrical engineering, and CS. The entire premise being "you type a URL into your browser and it loads the requested webpage".

Each textbook in the series at the depth of a bachelors degree, split by relevant subject including physics, chemistry, biology, neuroscience, materials science, electrical engineering, hardware design, kernel and device driver design, systems level networking, and all the remaining browser engine, VM, security, and webdev stuff.

To keep the task at least theoretically tractable cut off at the NIC in the local box.

Who knew you could premise a textbook on protein folding on "how a web page is loaded"?

great idea! i knew a physics professor once who developed an entire semester using "how racecars work" as his guiding light. i think he wrote a textbook, but i can't find it.

not: the physics of racecars, that is explaining racecar dynamics given physics fundamentals, rather explaining the fundamentals using racecars instead of spherical cows. i think not very different to first semester analog ee courses using hifi audio as the basis.

You may say that as a joke, but looking back at my education, this is in part what I did later, after uni, when I thought about things to learn next. In my mind, I started explaining stuff in depth, and found the many many many holes in my apparently quite superficial school knowledge.

I think this would be excellent as a once-in-a-while exercise to teach the complexity of it all, after we went the other way to abstract the hell out of everything to make it more palatable. A reality check, so to speak, and as a check for everyone about the many holes in their understanding. Basically, in support of something that has been posted here more than once, this blog post: http://johnsalvatier.org/blog/2017/reality-has-a-surprising-...

In the last few years I read a lot of easily digestible progression fantasy on RoyalRoad. One frequent theme is rebirth or isekai on another world, one that's usually much more primitive. Often the main character starts bringing earth technology ideas into the new world. And every single time it is obvious that the authors very, VERY severely the complexity and difficulty of even the smallest thing that we take for granted. My favorite blog post describing an extremely simple product, and how large and sophisticated its supply chain is: https://medium.com/@kevin_ashton/what-coke-contains-221d4499...

To me, this shows that some more awareness of how much complexity there is in things might be valuable.

Being more aware of the connections across our abstractions also helps finding what's missing. It could also help to find optimizations across abstraction borders, instead of limiting oneself to only looking within ones favorite abstraction layer.

> My favorite blog post describing an extremely simple product, and how large and sophisticated its supply chain is: https://medium.com/@kevin_ashton/what-coke-contains-221d4499...

is walled. Is there a way to read the article without extra steps?

nevermind the article doesn't even exist anymore

Your version of the URL is missing the last few characters!

https://medium.com/@kevin_ashton/what-coke-contains-221d4499...

The last string part is, again and to be sure since the visible URL is shortened by H, what-coke-contains-221d449929ef

I think you copy-pasted the visible URL, which on HN is unreliable. Long URLs are shortened for display, you need to use the underlying link.

> is walled. Is there a way to read the article without extra steps?

Archive link: https://archive.md/PPYez

> Your version of the URL is missing the last few characters!

oh, oops!!! yes I did copy and paste it, I typically do that when quoting parts of comments. I didn't realize it truncated the URL lmao

> Archive link: https://archive.md/PPYez

thank you!

The only good answer here is "breadth first, or depth first"?

If they say "depth first", go all the way down to electrons bumping into each other on the wire.

It's an "I'm so smart, are you so smart" question, the only right approach is a smart aleck approach.

I work on web browsers for a decade+ now, and worked on network protocols for a decade+, and it's not a question I'd consider in any way high signal. If this particular area matters to you, ask a problem in the space. "Web page X doesn't load, figure out why". It shows you way more than "please regurgitate every detail"

That's exactly the point - sometimes too much detail or focused on wrong thing can ve distracting.

But having a frontend engineer aware of Safari on iOS or macOS vs assuming chrome compatibility, or a SRE aware of abusive UDP http2 / h3 stuff...

Again the beauty is in the flexibility and scope of something normies take for granted, and this video delves into some of the esoterics of this topic!

Yep, having a video explaining it in detail is great. It's absolutely interesting. (I mean, I would say that, wouldn't I ;)

My objection was to using it as an interview question. An broad open-ended question like that without any guidance what the interviewer cares about is a signal of either copy-pasted hiring practices, or a working culture that doesn't know what it wants. It's also a question with little signal in the noise if you don't give a bit more clarity.

I mean, fundamentally I'm happy if other companies use the question, more good hires for me, but in terms of "we should really raise the bar on this interviewing thing" I still object :)

It's only low signal if you directly score the answer based on presence or correctness of specific details.

Forget about that. The question is an open ended and adaptable tool to get a huge amount of information about knowledge, verbal ability, audience awareness, and all sorts of other directly relevant skills. No trick questions just "teach me everything you know about this subject" and then work with the other party.

That said I suppose there's no reasonable expectation that (for example) most firmware developers necessarily know anything whatsoever about loading a web page. But anyone whose touches network (ie most) should know something.

In former times, this used to be “Describe what happens between pressing a key on the keyboard and the corresponding letter appearing on the screen”. You can go into all sorts of interesting tangents with that topic.

i literally am the inventor of this question.

it's an awful question.

You are! Amazing, I was asking it before you were a on HN. I guess I would have though you'd mention it before? https://hn.algolia.com/?&query=Describe%20user%3Ajiveturkey&...

I started asking it over 25yrs ago…

Never specified the level of detail I wanted but left it open to the candidate…depending on where they went it leaves plenty of room for follow on questions to probe their knowledge

Yes. I invented this question approx 25-26 years ago. At the time it was a pretty fair question. Today it's a trivia knowledge kind of question. You will be "impressed" by a candidate that can use up the entire interview (very possible) answering it, but that proves nothing about the candidate. Unless of course the job is writing system documentation based on personal knowledge only.

i change my HN identity once in a while. this is the longest run for me.

would be fun to do this while stepping through chromium w/ a debugger.

It depends on the role. I suspect the majority of software developers have no clue. The majority of C, C++, C#, Rust, Java, Python people, really don't know, unless their job deals with the browser. Most game devs don't know.

"All of the amazing things that modern computers do, are based on just ability to compute any computable function." - Ben Eater

source: https://youtu.be/AqNDk_UJW4k?list=PLowKtXNTBypGqImE405J2565d...

at 13:44s

The button on your keyboard is depressed, making a connection between two wires. The circuit is made and interpreted by a microcontroller, which differentiates which key press was made based upon…

What? You said excruciating…

How are you requesting the website? Are you typing into the address bar? Of which browser (and what version)? On which operating system (and what version)? On which hardware platform? Have you configured the address bar to also work as a search bar ("omnibox")? Are you typing a domain name or an IP address? Do you have any entries in your hosts file for the domain name? Is the machine configured to resolve hostnames on the local network? Is the domain name one of those? Are you connected to a network? Is your network connection over Wi-Fi or Ethernet? (or something else?) What is your network card and how is it connected? (If USB, which port? If PCIe, which slot?) What other network interfaces do you have? What access point are you using? Do you have a network switch, and if so, which one? Is your internet service over fiber or cable? Which ISP are you using? Gimme a second to look up their peering...

I could go on but :)

Is there an "explain like I'm 5" version of the answer to this question? I have read "The Internet explained from first principles" [0], and while I think I understood most of it in theory but, it didn't help me understand in practice.

[0]https://explained-from-first-principles.com/internet/#prefac...

"excruciating details"? I don't think I would ever get to the network if you asked that. I would have to cover how you pressing "enter" gets encoded by the keyboard, how the USB bus works, the interrupt processed by the processor, how that context switch happens, how the next instruction is fetched and decoded by the CPU, how the code for the interrupt gets read, TLB, L1/L2 cache, pipelined execution, microinstructions, transistors, ...

Depth-first doesn't work when the depth of reality is basically unlimited.

Is this the same guy talking about the same topic?

https://www.youtube.com/watch?v=p0lFyPuH8Zs

I need a transcript as the FOSDEM website didn't provide a one (404 not found), but youtube has.

Yep, it's the same talk, slightly updated.

Reply on news.ycombinator.com