Hacker News Reader

Comment by justin_oaks | Hacker News Reader

The part that resonated most with me is "Show things that are normally hidden".

Tools that do this make things clearer almost immediately. Consider the developer tools in a web browser. Do you remember the "dark ages" before such things existed? It was awful because you had to guess instead of seeing what was going on.

Tools like Wireshark that show you every last byte of network packets that it has access to AND parses it to help you see the structure. This isn't just for debugging networking data; it's hugely beneficial in teaching networking concepts because nothing is hidden.

This is also one of my favorite things about open source software. I can view the source to understand what's causing a bug, to fill in knowledge gaps left by the documentation, or just learn more about programming concepts. Nothing is hidden.

View on HN

The game development equivalent of these is renderDoc. I was stunned when I learned its existance for the first time.

I’m about to embark on my first real and truly massive graphics project. Thx for mentioning such a cool tool.

Wireshark is great but it does not show you every byte the network carried. For example it never shows Ethernet preambles, only sometimes shows Ethernet frame checksums, and never shows interpacket gaps (which are a required part of the Ethernet protocol).

So yes it comes close but it just goes to show you, there is always more detail hiding somewhere!

To be fair, the preamble can be easily considered just an electrical signal due to its objective (sync) which doesn't affect how the network works.

Yes, the toughest "hidden things" problems are pulling together data that is related, but not part of the same system. In this case, Wireshark can only show you what the OS gives to it.

In the article, it was pointed out that DNS caches can be hidden. They're especially hidden when they're upstream and in another computer!

Are the Ethernet frame checksums even visible to Wireshark, which hooks into the IP layer? would some of the ethernet stuff be only visible within the ethernet card itself, not to the software stack?

Sometimes they are. It depends on how the capture was generated. If you look in the options of Wireshark there is one to detect bad checksums, so clearly there is a way to capture them. Here is one such way: https://stackoverflow.com/questions/22101650/how-can-i-recei...

This can be used to detect partially-bad network cables, because there is no reason you should ever receive a bad FCS.

Wireshark works at the data link layer (L2).

Well...

ethhdr only has 14 bytes: 6 for dest mac address, 6 for source mac address, 2 for ethertype (e.g. IPv4 vs IPv6).

Any bad checksums that Wireshark can detect are predominantly at the transport layer (L4; TCP / UDP).

You'd have to explicitly turn on FCS, which I don't know if you can even do on, say, Windows.

Wireshark has support for lots of interfaces, it's just a consumer for OS/driver supplied data in this aspect.

See eg this bit about wlan for some of the complexity: https://wiki.wireshark.org/CaptureSetup/WLAN#link-layer-radi...

It doesn’t show electrical signals either. That’s a pointless nitpick if you’re developing anything beyond an Ethernet implementation.

My biggest problem with Wireshark is that it can't do anything with HTTPS traffic - which is most of the traffic I'd be interested in. I understand that's kind of the point of HTTPS, and a MITM proxy with cert replacement is somewhat out of the scope of Wireshark, it still limits the usefulness of the program.

Check out Charles Proxy for mac. It’s great.

it can, just not as easily as MITM proxies: https://wiki.wireshark.org/TLS

This method is really simple for Chrome, Firefox and other OpenSSL apps: https://wiki.wireshark.org/TLS#using-the-pre-master-secret

You set an environment variable to instruct the app the write a file that Wireshark can use to decrypt its traffic, and change a setting in Wireshark to use that file, and that's it.

You will even be able to see decrypted WebRTC traffic.

Wireshark will also frequently show the wrong checksums for IP, UDP, and TCP. Specifically when computing them is ofloaded to the network card.

My dream is to make everything visualizable at runtime. I think all of computing becomes very simple and much less complex if we can do this.

We are visualizing things in our head already. And any explanation of anything in computing is a diagram. But we have zero diagrams when coding.

Just dynamically instrument all code to send messages to a GUI.

Clojure does pretty well. See https://github.com/nubank/morse, https://docs.datomic.com/cloud/other-tools/REBL.html, and https://vlaaad.github.io/reveal/.

It's one of the areas that homoiconicity helps: code is data, data is code, so visualization tools can work on both sides.

>My dream is to make everything visualizable at runtime.

Check out demos of the old Lisp Machines, [1] is a brief overview demo, [2] links to a timestamp with a view of some simple diagramming, but I’ve seen TI-Symbolics beasts routinely display complex relationships in Lisp code on their massive (for the time) bitmapped screens. The limitation was the end user managing the visualization complexity.

With open source llvm, clang and similar making available abstract syntax trees and even semantic analysis and type checking results, LLM’s assisting with decompiling binary blobs, and modern hardware (goggles, graphics cards, and so on), I sometimes wonder how close we can come to reproducing that aspect of the Lisp Machine experience on open source operating systems.

[1] https://youtu.be/o4-YnLpLgtk

[2] https://youtu.be/jACcgLfyiyM?t=43m52s

'"Show things that are normally hidden". Tools that do this make things clearer almost immediately.'

At least with all the DevOps shit going on now, it seems that many of the tools increasingly hide things. And the gurus who had that knowledge and could teach you are now concentrated in those tool companies instead of in your org.

This is one thing I love about Magit for Emacs. The UI is really clever and slick—maybe the best Git frontend that I've ever used—but the way you interact with UI is by toggling flags and options that actually map to the underlying command line Git arguments. I can seamlessly hop into the command line and feel right at home using it directly.

On top of the UI having very tight and visible relationship to git CLI commands, there's also a log buffer showing you exact commands executed and their outputs, and it's only a single $ press away.

Yes, text based UI's like magit offer this kind of simplicity without sacrificing cli speed.

Man, having spent waaaay too long troubleshooting errors where it's not remotely clear what part of the config the program is even consulting makes this hit home.

This is what I like about SQL databases. They frequently have tables that you can query to find system information. This makes it easy to explore the system within the system.

The proc filesystem on Linux is philosophically similar. It allows you to understand processes by working with files.

On the contrary, you don't need the source, and in some cases it may even be misleading in many ways, when you can look directly at the instructions the machine executes. Tools like disassemblers and decompilers would be equivalent to what you speak of.

Nothing stops you from doing this with open source software either, though; making something open source is strictly an increase in the information available, and at least to me that seems in the spirit of "show me things that are normally hidden"

Yep. Mozilla developer tools basically launched my career.

https://news.ycombinator.com/item?id=37467078

Is there a way to get Awk to emit a non-terse version of the script passed in? ie awk '/test/' -> '{ if($0~/test/){print $0} }'

Sector editor!

(Or back in the day, looking at the source code because we ran uncompiled stuff in Basic and whatever and that was pretty cool)

That's one of the advantages of using programming languages where source code is distributed (e.g. Python, JavaScript, PHP), not compiled binary artifacts (C/C++, Java). You can see the source. You can even modify it and run the modified version without compilation.

It's also awesome to use Java IDEs that can show both the bytecode of .class files and also perform decompilation.

Reply on news.ycombinator.com