2,461 karma · joined October 27, 2013
The datasheet only gets you so far, especially for undocumented hardware quirks, and the vendor is in a much better place to understand what's happening and fix them. It's an active process of reporting bugs, investigating them, and getting fixes merged upstream into the vendor's BSP, at the very least. This requires active maintenance, which means engineers on the vendor side. That's where the support contracts kick in.
Upstreaming to Linux on top of all that means that you actually need to support and validate the set of features your chip claims to support. That means more thorough test and validation, code that meets the standards of the kernel, and more engineers to actively maintain the code. Not to mention the language barrier for Chinese chip vendors.
Of course, any upstreaming efforts are great and I think things are trending in a good direction. Just pointing out that there are reasons things are the way they are. Fabbing a chip that isn't completely buggy and people want to buy is already a small miracle- better software support can always wait until the chip becomes a hit.
The customer then needs to figure out what they want to do: Take the kernel blob as-is? Get the vendor’s kernel source and try and modify it for their specific needs? Rebase it to a newer kernel? The customer typically wants to just get enough support to ship their product; they’re not in the business of maintaining some other company’s kernel patches. They’ll do it if they have to, but the engineers will grumble about it the entire time.
Thus, getting support landed into Linux upstream is primarily the task of the chip vendor. There’s a good business case for it, as better and widely available software support will help sell the chip. But it’s also incredibly expensive to write all that kernel code and take it through the process of upstreaming. A lot of this stuff won’t meet the standards of the Linux kernel without lots of work.
The ARM SBC market flips this, where you have a bunch of enthusiasts, and then a smaller market of businesses wanting to use them in their products. That can provide enough momentum for a community-driven upstreaming effort.
The laptop docking story is also significantly better. Waylands libinput lets me assign mouse accel and keyboard repeat rates/rebinds to specific devices, and they always get applied on hotswap, no custom udev rules needed. For audio, Pipewire is sensible and Just Works. Kanshi works sensibly in a way that I was never able to convince dockd to.
Wayland is what brought me from “I like Linux for a work/coding environment, but it’s too fiddly for a daily driver” to finally consistently using Linux in a personal and professional capacity.
I commented about it a while back here: https://news.ycombinator.com/item?id=34374725
"How a computer works" is very broad, and covers a lot of different disciplines. Like everything else in computing, it's layers of abstraction, and there's a lot of depth you can go into at any level. What you're describing is basically an ECE undergraduate curriculum, but it's very possible to get a "tour" of various layers in less time than that without going to school for it.
There's a lot of different ways to approach learning, and the bottom-to-top tourist approach is certainly one way to do it. I personally would not recommend it, as you're going to run into lots of different concepts that will take time to digest, and some of the tooling will be rather opaque or foreign, without a mentor to guide you. Another approach would be top-to-bottom, where you do some C systems and kernel hacking and work your way down from there. If you the art of programming and Computer Science, that might be a better fit, but there's also lots of arcane stuff in Linux and x86 that can be demotivating. The approach I advocate is usually the "maker" one, where you hang out in embedded systems land, and come up with projects you want to make. The benefit of embedded projects is that your projects will be physical things that interact with the real world (which may or may not be actually useful). That lets you explore a breadth of topics as needed, but is a bit adjacent to your goal of "knowing how a computer works".
Re: geohotz's curriculum, have others actually done his course? The curriculum looks interesting, but laser focused on bringing up a CPU in Verilog and bootstrapping the OS in C. I'm not sure how easy the course would be without at least some Verilog and systems programming background. 12 weeks seems very aggressive, and you're gonna end up missing concepts if you try to do it at that pace. The course I usually see recommended is nand2tetris, but I haven't tried it myself.
For foundational curriculum, I would probably recommend:
* The CSAPP Book (https://csapp.cs.cmu.edu/) - The introductory systems programming book/course. The labs are really good; do them! You could probably spend a year just with this course and come away satisfied. It'll have you interfacing with computers in C, learning about how cache works, writing your own malloc, doing some networking code, etc.
* Circuit Design - Optional, but some electronics knowledge doesn't hurt, depending on how far down the stack you want to go. Don't need to go into the fancy stuff, but should at least know enough of the basics to understand how transistors work, and how they can be used to form gates.
* Digital Design / FPGA stuff - I don't have a good rec for this. Nand2Tetris? There are some interesting projects out there that make FPGAs more accessible, like fomu and the reverse engineered open lattice toolchains.
* Computer Architecture by Hennessy & Patterson - Guides you through the concepts of a CPU, and has you writing your own MIPS CPU in Verilog. ARM is cool and all, but for a toy CPU, MIPS is a little more manageable, IMO. If you try to go straight to comp arch without some base level of digital design/FPGA/Verilog experience, you will probably have a bad time or end up with some gaps in your knowledge.
The main advantage of digital in cine is that you can shoot and reshoot without dealing with the cost and logistics of film. But it’s Nolan’s summer blockbuster in IMAX format, I think they’ll be alright.
With software, it costs nothing to just git clone and build for your personal use (which also presents its own funding challenges). For hardware, there's gonna be a chunk of people who just want the product/functionality via clones, and you need to be careful to keep the community/project to grow sustainably. IMO, it should be along-side and not in spite of those clones, but you can see the frustration.
If you want to try it: https://github.com/Hylian/atkinson-monolegible
Where I do see it being handy is with mundane tasks or those "tip of my tongue" queries. I was doing some tedious mapping of enums to other enums today, and that's one of those problems where you can script it out in vim/sed, but then need to spend a while double checking it against the spec anyways. I'd love to ask a bot to do that for me. Similarly, the easy stuff like "how do I do [common git operation] again?", it seems to do great on, since there are canonical answers to that stuff.
I've been generally uninterested in the ChatGPT craze, but I did check out perplexity.ai when it popped up on HN a few days ago. It did seem genuinely useful for these types of queries, and it sources its answers with web links. I could see a space for that type of tool, as long as the hairier parts of the chatbot can be filtered away.
I had issues with just DNAT following some ui forum posts, and I think it's because I'm using switch0 as the interface, with devices across LAN ports on my EdgeRouter. The SNAT masquerade was the key to getting the replies from the pi-hole routed properly.
I was previously using my EdgeRouter as the DHCP DNS server, and using DNS Forwarding on switch0 w/ dnsmasq to forward queries to the pi-hole. With a large cache, this avoided a couple hops for every DNS query on my network. However, this meant that I couldn't see which IP was making the query in the pi-hole query logs, so I've flipped it so that the pi-hole is the DHCP DNS server, and the pi-hole queries the EdgeRouter, which then forwards the requests to public DNS w/ caching. I then assigned every device a '.local' domain in the pi-hole Local DNS tab, which lets pi-hole displays a friendly name for each query in the log.
For completeness sake, here's everything I needed on the EdgeRouter. The EdgeRouter was setup with no VLANs, and with all LAN ports switched w/ a single subnet (switch0 interface exists in the dashboard).
EdgeRouter:
* In the bottom left "System" pop up drawer, set "System domain-name" to "local" (or whatever domain of your choice, like "lan" or "home").
* Services > DHCP Server > [Your DHCP Server] > View Details: Set DNS1 to your pi-hole IP (make sure it's statically mapped!), set Domain to "local"
* Firewall/NAT > NAT: Follow above guide
* Services > DNS: Enable DNS Forwarding for switch0, set appropriate cache size
* Config Tree > service > dns > forwarding: set name server to public DNS server of your choicse
Pi-hole:
* Settings > DNS: Disable all external DNS servers, set custom upstream DNS server to the EdgeRouter
* Local DNS > DNS Records: Look at your EdgeRouter DHCP lease list, give everything you care about a static IP assignment, and then give them ".local" DNS records in pi-hole. SSH-ing in and editing `/etc/pihole/custom.list` may be faster, as pi-hole seems to bring services down and then back up for each entry added via the web UI.
ngfts.lge.com
us.ad.lgsmartad.com
lgad.cjpowercast.com
us.info.lgsmartad.com
aic.recommend.lgtvcommon.com
aic.homeprv.lgtvcommon.com
aid.rdl.lgtvcommon.com
aic.lgshopsvc.lgappstv.com
^aic.*lg.*
us.emp.lgsmartplatform.com
snu.lge.com
us.lgrecommends.lgappstv.com
api.thetake.com
us.lgtvsdp.com
aic.service.lgtvcommon.com
lgtvonline.lge.com
(\.|^)gracenote\.com$
(\.|^)prehook\.com$
raw.vidyard.com
(\.|^)vidyard\.com$
(\.|^)wistia\.com$One thing I've been feeling lately with internet-connected home devices is that I'm being pushed towards products from big tech companies like Google. Google might mine my data still, but at least they'll ostensibly do it competently and securely. Not a great feeling. Despite not really being in the Google ecosystem, maybe my next TV will be one with Android TV support. On the smart home side, I did get into self-hosting Home Assistant recently. I didn't expect much from it at first, but it's remarkable how liberating and useful it feels compared to the one-size-fits-all approach of Google Home or HomeKit. I'm actually excited to try and get some more smart home devices now, beyond just a few lightbulbs.
As for the pi-hole setup itself, it's working great. It's a great backup to locally installed adblockers, and I have my EdgeRouter as the primary DHCP DNS server, which makes requests to pi-hole. Both the router and pi-hole have caching, and my DNS query latencies are good. Pi-hole also has a nice interface for pointing `.local` domains to local IP addresses, which is much easier than messing with dnsmasq settings on the EdgeRouter.
I still love embedded, and I think it's great for all the reasons mentioned in the thread. As for your question, firmware is quite broad, and there are lots of different routes people take to break into it as a career. At its core, it's a lot of writing systems code, or feature code that runs on constrained systems. Something like CS undergrad systems programming + OS or equivalent should be enough.
After that, I would make sure you have decent breadth: knowing how to read datasheets, bringing up parts, writing a linker script, using peripherals, grokking some standard protocols like I2C/SPI/SDIO, basic circuits knowledge, basic comparch knowledge, etc. The nice part of all this stuff is that you can totally get there with just hobby projects and Adafruit boards. If you find yourself using the batteries included in the Arduino or Adafruit libraries, maybe push yourself to write more of it from scratch. eg. write an SDIO driver to read from an SD card and parse a FAT filesystem.
People with solid software engineering backgrounds are pretty valuable in embedded, so I'd emphasize that as well. Knowing that git exists, unit tests, API design, CI tooling, etc. If you're a Linux kernel hacker, embedded linux is also kind of its own separate world, so that might be a good direction if you have experience there.