Show HN: I'm writing a free book called Computer Networks from Scratch
networksfromscratch.com
networksfromscratch.com
Ethernet, and 802.11, are both based on the concept of a singular channel shared by many individuals. When Alice talks, she says 'I'm Alice calling Bob, can you hear me Over'.
Alice is the source address. Bob is the destination address. Over is the stop code/stop bit signifying the end of frame. The payload is 'can you hear me'.
-------
This is everything up to layer 3. But if you wanted to send a radio/telegram across the country, you need one more layer, IP4 or the ARRL protocol (American Radio Relay League meets on the air at predestinated times to pass messages to each other on Ham radio frequencies. It's a human network)
I'm Alice, calling the Eastern Operators. I'd like you to relay a message to Bob for me. The message is 'Can you Hear me', Over.
This is a radio frame like all others, but an additional layer has been added. The radio operator has its own protocol built on top of the Frames.
------
Dave may overhear all of this chatter, but Dave knows to stay off the channel (otherwise he will cause interference). Only if Dave is addressed specifically would Dave respond (Ex: This is Alice, calling Dave. How are you doing? Over)
We can see that the operator is itself just another person on the network. The operator may have a message from the rest of the world.
This is the Eastern Operator calling Alice. I'm relaying a message from Bob. The message is 'I got your message Alice!'. Over.
This is how a router may take a message from the greater internet and pass it to your local network.
What would be the Bob-Alice-Dave equivalent to replacing the network hub with a switch?
So I guess cellphones instead of one-at-a-time walkie talkies, since everyone on the cellphone network has a unique connection.
At least in a wireless context, CDMA and TDMA are flawed examples because, CDMA uses fancy math to split communications over multiple channels and TDMA cuts the channels up into timeslots. You don't get full bandwidth allotment and utilization.
To make a wireless switch, you'd need to have PTP optical/microwave/radio links which have highly directional antennas. This would give you full-bandwidth per user.
If I'm not mistaken, the radio example uses both frequency division and time division. You're on a particular channel (typically frequency, although AM exists too), and only one person can talk at a given instant, or else you get a collision which results in probably garbled output. (There's also space division at play, since radios are not infinitely powerful.)
If the relay hop is also done over radio, then it's most likely achieved by hopping to dedicated frequencies for backbone comms, ideally with a separate antenna per channel so we can listen on all of them simultaneously. (You could constantly sweep the frequency space, but it's simpler to just have multiple antennas / radios.)
A (modern) real-world example is that I can use my phone as a wifi hotspot. It relays packets to/from its upstream link (4G or even wifi again). (It also presumably behaves like a NAT, but that's an implementation detail.)
When I write software I think of objects and messages in a similar way.
full duplex ethernet circuits (eg: a basic cheap 10Gbps 1310/LR carried over two $30 transceivers and a six foot fiber patch cable between two pieces of equipment) and a half-duplex TDD/CSMA regime like 802.11b/g/n/ac/ax are not the same thing.
yes they both enable ethernet connectivity between devices on fabrics of things that can see and arp each other.
the ethernet 10GbE might be shared by many individuals. but it's full speed both directions. one is half duplex, the other is not. one has contention on the shared media airtime, the other does not.
they are both possibly shared in the sense that many peoples' traffic may be carried across the 10GbE full duplex circuit, the the real world implementation is very different.
the real world implementations of FDD point to point (FCC part 101) or similar radio systems are closer to wired ethernet in concept, though with less capacity.
however I do agree with your general point that more people should familiarize themselves with the limitations and usage methodologies for half duplex IP data mediums and the best real world implementations.
I probably should have clarified that when I said "Ethernet", I meant classic 10 Mbps Ethernet that today's kids probably don't know about. The Ethernet hub acted like a bus: anything one connection said was broadcast to _ALL_ other connections.
Of course, today we have far faster point-to-point switches. But it helps to remember that the Ethernet-protocol was built in the days of old: half-duplex shared communication broadcast to everyone on your hub.
-------
Today's wired 1Gbps or 10Gbps ethernet are way way faster than half-duplex channels that 802.11 / Wireless technologies are forced to use (after all: the physics behind 802.11 and walkie talkies remains fundamentally the same)
There probably isn't a need for CSMA/CD systems in Ethernet anymore (since today's networks are all dedicated wires). But still, the CSMA/CD system exists for a reason, a legacy of the old "shared ether network communications" that gives rise to the name "Ethernet". (One Ether that everyone on the network can hear simultaneously, like a walkie talkie)
Looking at CSMA/CD more specifically, as well as "full-duplex Ethernet" (which are 100Mbit, 1Gbit, and 10Gbit modern Ethernet), it seems like CSMA/CD is no longer used in any full-duplex Ethernet Protocol.
I'm not 100% sure about that, since its outside the scope of my old studies. But it makes sense to me. It seems like "full duplex Ethernet" can only work with point-to-point communications. If you have point-to-point connections between exactly two stations, there's no need for carrier-sense (CS), multiple-access (MA), or collision-detection (CD) protocols at all.
CS: Wait for the current person on the walkie-talkie to say "over" before you yourself start talking.
MA: Walkie talkies: many people can access the same channel.
CD: If two people accidentally start talking at exactly the same time, they need to independently try to restart-talking. A random-number generator with exponential backoff is common for this sort of thing IIRC, though I'm not keen on the exact details.
There's a reason why another name of Layer 2 ethernet network is "broadcast area".
https://www.amazon.co.uk/Code-Language-Computer-Hardware-Sof...
I feel tempted to buy the current edition, but also to wait for the second.
There is still so much of the tech stack that hasn't changed since this book was published in 2000 - TCP, IP, DNS, binary, logic gates, etc. But there's also much that has changed that could be written about - virtualisation, containers, wifi, fibre, cloud infrastructure, USB, GPUs.
That way Dave knows to ignore the message if it doesn't start with his name (or "Everyone?" for broadcasts).
One callout: a table of contents is needed for the online version. One that links directly to the chapters. And if one exists, it needs to be easier to find because I couldn't locate it.
That's awesome!
>I wish I had had a book like this to make that change faster.
That's the goal.
I definitely need a table of contents, but I was planning on adding it once I got a few more chapters done. Based on the comments though, I should probably add one quicker than that.
Just yesterday I couldn't fall asleep because my stupid brain thought something like "What information would I need to fully grasp the topic 'programming'?". Since I am a visual thinker, I quickly started to draw a mind map in my head that basically started with the history (starting with Alan Turing), going through electrical engineering, went through different layers and finished in different programming languages. The languages alone branched out in a pattern so large that my head startet to hurt. It included different languages, concepts, architecture patterns and compilers. Each one of those steps is important and you will find people specialized in any of those which will tell you that their part is the most important.
I am a web developer myself and I try to learn and improve every day. But I also think to be really good you have to focus on a few specific parts of the process and not try to be an expert in everything there. That grew way too complex quite a while ago and trying to understand everything usually only makes you mediocre in everything but good at nothing in particular. If a web developer does not know the difference of a GET and a POST I would seriously suggest to evaluate their choice of profession. But for example the last time I've had to care about a subnet mask was 11 years ago and I would not ask anyone I recruit as a web dev to know anything about this topic. You have to draw a line somewhere. My bottom line in all this is that you should have at least a general understanding on the whole process, but you only need to be an expert in a few of them to be really good.
I'm curious what specific mistakes you're thinking of that application developers make when they don't understand computer networks. In my limited experience, the rudimentary abstraction "messages are sent, they're not instant and they might not make it" has generally been sufficient, aside from DNS issues. I learned some networking in college (CS241 at UIUC) and it was really interesting, but I feel I've forgotten almost everything from lack of use. Maybe I've done terrible things as a result. I think I and many others could really benefit from a list of common pitfalls by networking newbs.
There are also simpler issues. Not knowing the difference between GET and POST, not understanding what a header is, how parameters are passed etc...
Then there are the problems in the middle like not knowing how to correctly configure an HTTP client library.
The purpose of writing the book though is to help people develop a good mental model of the system, so that they don't need to memorize a list of all the things that can go wrong.
Adding a few examples to the intro wouldn't be a bad idea though!
People building games, robots, trading systems, distributed databases, CDNs, etc could be really limited in their capability if they didn’t grok networking concepts on a deeper level. It also tends to tie in heavily to other systems concepts (things like zero-copy, or even basic elements like ring buffers).
I think you might also underestimate just how little knowledge a lot of people have. You went to a legit uni and took a legit course.
The interview question OP asks is an old favourite I used to ask too. A lot of people answered something like “clicky-linky-page-rendery!” So a bit of education on the many layers below that doesn’t hurt.
I once blew the minds of a room full of senior engineers by setting up a server which responds with the headers and different chunks of the body at different times, and so the content appears sequentially in the browser with a delay. They literally did not know you could do that.
Futurama had this figured out:
"One beep for yes, two beeps for no."
"Beep-beep"
"That means double yes!"
A friend and I are writing a book on Low-level programming. We are still working on Part 1. The plan is to make it available (along with programming exercises) for free so anyone interested can benefit.
I would love to hear about your plans/experiences on how to spread the word that such material exists for free. Ideas for ensuring the material is found by maximum number of people and benefits them.
I'm definitely a bit guilty of this.
OP do you plan to cover stuff used in IOT? Treating HTTP as a magic black box kind of works fine until you have to consider MQTT, COAP etc.
I’ve done some IOT work so it is something that interests me.
I’ve struggled to find a book that doesn’t just hand wave layer 1 and assume layer 2 is ethernet everywhere.
It's not all that network related.
WiFi and xray transmissions is of course covered in most books though at least in terms of its capabilities.
What more do you want to know about level 1 and 2?
It's insanely rare in my industry to encounter anything that isn't ethernet, hell even ethernet with MTUs below 1500 are fairly rare outside of VPN provisions. Sure there are some, especially in the high end areas, but it's so rare and tends to eventually present as ethernet.
Far more use for almost everyone is an understanding of Ethernet, IP(4+6), TCP/UDP/ICMP and how routing works (the concepts more than the specifics - things like how traffic can route asymetrically, how packets can be sent on different routes, etc). You could include other protocols like LLDP and arp (maybe LACP as part of the different routes), mention about how a traceroute will hide information when routers either don't decrement MTUs, or just stick the entire packet in an MPLS packet and pass it through the network, mention how just because your traceroute shows 20% loss at hop 7 it doesn't mean hop 7 is bad, and again how the ttl expired response may be dealt with with a low priority, or travel on a completely separate network, etc.
I would think all of that is far more useful than knowing that you need Manchester to keep your clock or how you use PSK to increase the symbols through your medium
It might be a little early to start spreading the word though, with only 3 chapters ready so far. Hopefully when it's closer to being done you'll be able to get some attention from HN
Releasing early was mostly a choice to help motivate me. As well as to get early feedback and proofreaders.
>It looks like this could be pitched at 400 undergrad ""basic coms and nets"
400 level undergrad is about the level of detail I'm aiming for (eventually). I've been using one of my old undergrad networking textbooks to do sanity checks.
>You can still sell a lot of print books with the pdf free online, so I hope that works out well for you.
Here's hoping!
I wish somebody would write something like the above for the "Modern" Internet.
>I read most of it and thought it went into great detail explaining the basics to the general audience.
Thank you!
>It also doesn't require any programming, well mostly!
Yep, I did have to throw in a bit of pseudocode for some of the microcontroller section.
In https://www.networksfromscratch.com/2.html
"Remember that when someone presses button 0 the output is set to -5v and when someone presses button 1, the output is set to +5v."
This should be 0v, +5v.
"We connect two output pins on the letter writer to two input pins to the input pins that were previously connected to the buttons."
This sounds really weird.
I tried checking the content with my kindle (a Kindle Touch paperwhite 6 inches) and the first chapter rendered pretty good, with images and text aligning as intended. However the second chapter did not render the first figure and beyond. Chapter 3 rendered OK, similar to Ch 1.
A few more feedback if that helps for enhancing the format even more: - Colors, and references to those do not work on a kindle. I only saw one minor use of colors (red, green) in Chapter 1 so that's not a blocker. - For some reason, the kindle browser doesn't put left and right margins on the page, so the text uses the full length of the device. - Links are rendered with a pale gray, sometimes hard to distinguish. - Figuring out why the Chapter 2 did not render would definitely help.
The reason why I recommend learning about Computer Networks from scratch is how fascinating the subject is. The way layers are implemented, their interactions, how the overall complexity is reduced by having layers of responsibility. This pattern can be seen in other areas as well: hardware -> operating system -> SDK -> programming language -> framework -> DSL. Ultimately from a Turing complete machine to seeing 200 FPS 3D FPS game with 150 other players on your screen.
I have a huge gap in knowledge for so many things that I interact with on a regular basis, but that I don't actually need to understand well in order to meet minimum job expectations.
This seems like a good book to address network basics. Sadly, I'm a huge procrastinator, but this seems especially important for me to address.
Books that have been in my backlog for years: Database Design for Mere Mortals, Designing Data-Intensive Applications, Building Secure and Reliable Systems, Writing a Compiler in Go, Crafting Interpreters.
All books that I know I need to read, but I just can't get around to it because there's no immediate reward for me, as someone who doesn't have to worry about any of these days in my day-to-day duties.
When sailing through the computer network topic, my great companion was Tanenbaum's and Richard Stevens's book. Although, I have to be honest that if I only read that book without exposure to the real-world problem, I might quit computer networks many years ago.
My two cents for the rest of the chapters would be: to make sure to provide a real-world problem and example in which anyone could have access to them. In my case, I had the privilege to build a real network (10k users) with FreeBSD as the router, proxy, DNS, mail, etc. With all the tooling available today, one can easily acquire these swiss army knives for networking. Sure, tooling comes and goes, but at least it can provide better abstractions to the students. At least it worked for me.
I have written, or been involved in the writing of, a few books. My e pertain every was that it is easier to start a book from the TOC.
I am assuming you will be using LaTex or something similar? Writing a book in anything other than a structured writing environment can be hell. Certainly avoid ms word like a cockroach should avoid a hammer.
Also: you need a table of contents!
If I'm not mistaken, it looks like there's a typo under the section "Binary is better?": it reads "...then R is also number 5" when I think it should read "...then F is also number 5".
I think a table of contents is warranted. Even if it just for 3 or 4 chapters.
> As it happens, 00101 is also binary for the number 5. If you start counting at 0 (A = 0, B = 1, C = 2, and so on), then R is also number 5.
R is 17, not 5. You might want to reword it. Stopped here to ask you this and haven't read further yet.
https://youtu.be/hFsjbBfFyLg?t=503
It was meant to be a Django tutorial, but I wanted the audience to get an idea about how HTTP works.
May be it's just me and the kind of people I have been meeting who do not seem to have as much patience and time and would prefer getting hands-on experience quickly.
I can see how I might sound complaining about going through ~30 pages of information whereas full books with several hundred pages take much longer to read and this is relatively much shorter.
Tutorials are great if you want to learn how to use library X. I prefer books if I want to understand the problem that library X solves. Of course that's a sweeping generalization.
I remember there was a good article on the theory, especially the rewind part.
"This site is blocked due to a security threat."
The xkcd-style stick figures are particularly appreciated as a form of sugar to help the medicine go down.
P.S. Scott Adams has talked about using the same technique for the Dilbert comics.
I see you are going for the Thing Explainer style a la XKCD. That only works for a subset of learners.
A major hurdle is that you jump right in to “imaginary devices”. And these imaginary devices are rolling marbles of all things.
Perhaps a better way would be a historical look at how communication evolved. Smoke Signals, Flags on Ships, Telegraph, Radio, Switching Telephone, Packet Telephones
These are concrete things people can see and grok.
See “Don’t make me think” book. The essence is that “imaginary devices” is super effortful for the reader and hampers the gradual learning for beginners to acquire the knowledge