KnightOS was an interesting operating system | Hacker News Reader

KnightOS was an interesting operating system

drewdevault.com

292 pointsakalin60 comments

When I was a school IT admin, I saw students try a lot of creative things.

Some of the best, were thanks to KnightOS.

There are a lot of stories, but I might relate what might have to be my proudest.

The teacher had caught the students fooling around with BASIC to share notes, and for some reason thought that sharing notes about what they were talking about was counterproductive to the lesson, so asked me to erase the capability.

I didn't want to, but orders are orders, and I could only appeal all the way up the food chain to the principal, who barely managed to use his email on his best days.

So as I was doing as instructed, I noticed one of the calculators that came across my desk had KnightOS. This one hadn't been part of the note sharing, but had an interesting program on board that was still being written.

I called the student in excitedly, to ask them to explain, hoping I was right. I was. They were making a program that transmitted a minimal MIMEfile across the 2.5mm cable from one device to another. They were implementing email.

This was the first C-compiler that they'd be allowed to touch. Their parents strongly believed that girls had no place in IT, and did everything they could to keep her away from the computer at home. Her calculator with KnightOS was her only outlet.

When it came time to hand all the calculators back, I casually mentioned the program she wrote to one or two of the students.

Within the week, everyone in the class was running KnightOS and sending each other emails of notes in the class.

When the teacher came back and complained, I then had the power to appeal to the principal that the teacher was complaining that her students were using email, using a program that a student had built herself to send email.

When the principal heard that, it wasn't note passing anymore, because he knew what email was. It went from potential punishment, to the girl who wrote an email program getting called up in assembly to be congratulated in front of everyone.

I didn't really keep taps on anyone, but from what I've heard, that girl now works for a game development company in town.

I wanna believe this story, but it's almost too perfect to be true :)

A simple subset of MIME is super-easy to do.

I have plenty of other _boring_ stories to offer. Kids are creative. The moment you try and bring a hammer down on what they can do, they'll show you that no technical solution will stop them.

Thank you for this heartwarming story.

Damn, I'd love to know the things she has done since then, there must be some pretty interesting stuff among them.

I feel like constraints are where developers get to shine. On a system with (relative) unlimited resources, many developers seem to be able to complete the same tasks. But when there is scale involved or some constraint (like 4 mb of storage) you get to find out who is willing to learn what they need to learn to get the job done.

It's the litmus test: How much of taught abstractions do you really understand (can you adhere to the underlying coding practice without spelling it out in (costly) code? Do you really understand a runtime environment and can you make your basic tool, the compiler, optimize for your use-case?

Back then, when I was young and stupid, I agreed to develop a Command & Conquer like game in j2me with a deliverable size of max 120k. I did, but that required lot of all-nighters and a severely bruised and corrected ego..

And, as the senior dev, I got to listen to very smart recently graduated devs who did not know shit (as I did when I started). Best management lesson ever...

The nice thing about constraints is that they know what they want.

That's a very pithy way of putting it

There are a bunch of styles of X-Driven-Development, and I honestly think we should add the notion of Constraint-Driven-Development as a first-class citizen to that mix. CDD.

Start by specifying the constraints, including hardware, software, formal specification, and business rules. Put them all into a Constraints Document. Then tell the engineering and design teams to work together to make the coolest, most creative product they can, without violating the specified constraints. Use Agile, rapid-iteration methodologies if you like, within that framework.

Then evaluate the outcome, and decide whether to iterate by updating the constraints document and repeating the process. Or maybe you’re happy with v1.0 and no iteration necessary.

Constraints drive both innovation and quality. I wonder if the process can be viably formalized in such a manner.

As Goethe wrote: ”In der Beschränkung zeigt sich erst der Meister” (“the master shows himself through self-limitation”).

One can see this as a form of constrained writing (https://en.wikipedia.org/wiki/Constrained_writing)

Demos (https://en.wikipedia.org/wiki/Demoscene) and The submissions to the International Obfuscated C Code Contest (http://ioccc.org/) also are examples of this.

My day job is web development but I love hacking on microcontrollers in my spare time. One project involved serving a static webpage from an ESP-8266. But how do you server an HTML file without a filesystem? There's probably a better way, but my solution was to gzip the file and embed it as an array of raw bytes within the program.

ESP8266 has basic filesystem support, have a look at this: https://arduino-esp8266.readthedocs.io/en/latest/filesystem..... You define the filesystem's size when you flash the ESP and there's a tool to upload files there. You should also be able to find example projects that serve a webpage read from SPIFFS. Edit: this is a fake AP for esp8266 that I wrote some time ago that has a captive portal that loads pages from SPIFFS if they're there, otherwise sends some default pages directly from code https://github.com/GTP95/CaptivePortalMatematico?files=1

Cool! I ended up switching to a raspberry pi for that particular project but I'll keep that in mind.

https://github.com/me-no-dev/ESPAsyncWebServer#respond-with-...

That whole ESPAsyncWebServer github project has a lot of great examples on ways to use the esp8266.

Also, if you want to connect to your esp8266 externally, you can setup nginx (maybe on a raspberry pi?) as a reverse proxy with TLS enabled: https://jjssoftware.github.io/secure-your-esp8266/. Probably a bit pointless if you only have one esp8266 to connect to, since the RPi could probably do it all. But if you have multiple IoT devices to connect to, might be worth it.

Do you mean like Apache mod_asis [0] which will serve a pre-formatted HTTP response, including the headers, directly from disk?

Then your HTTP server can speak very little HTTP (compare request line to preformatted "GET /file HTTP/", return 404 on mismatch, return preformtted response on match). Maybe also return nice error if request does not have accept-encoding: gzip.

0 - https://httpd.apache.org/docs/2.4/mod/mod_asis.html

As far as program resources go, on DOS, at least, it used to be possible to simply append a file to the executable, and the loader would put everything in RAM. (On the other hand, you do not need a generic web server to serve a single static page; then you don’t need a file system to store one file.)

This is still pretty common for things like self extracting archives on Windows.

  copy /b "%windir%\system32\extrac32.exe"+inputarchive.cab selfextract.exe

You can do that with most executable formats: MZ/NE/PE (DOS, Windows), ELF (Linux, BSD). Not sure about mach-o (macOS) though.

I feel like constraints are where developers get to shine.

Absolutely. And I think that's one of the ways you can tell that there is an art to programming. It's not just pushing buttons in the right order. (Something my boss doesn't understand because she took a computer class in high school and thinks she can do my job. "I know code.")

Computer science is a thing. Programming is a thing. They're not the same thing. And as we get into more abstractions, I feel like the programming side is becoming more an art form than the science it is presumed to be.

> Computer science is a thing. Programming is a thing.

Engineering is also a thing but that's vanishing too.

I always liked how rms says that programming is a craft.

If you like constraints, you'll probably enjoy https://en.wikipedia.org/wiki/Demoscene . Some productions have made it onto HN, like https://news.ycombinator.com/item?id=11848097 and https://news.ycombinator.com/item?id=14164907 or even https://news.ycombinator.com/item?id=9338944

Google Sheets is my constraint. Utilities.sleep()

When I was in high school, I always wanted to build a better OS for the TI-84, but I did not have the skills or attention span to do it on my own. When KnightOS came along, I got involved and was able to take on individual features or issues as bite size chunks. I was able to make real contributions to a project that resonated with me on a fundamental level. In the process, I learned z80 assembly, how OSs work, but also most importantly how to contribute to a real open source project.

Thanks, Drew.

Hey, I remember you :) thanks for all of your help back in the day. I was just thinking to myself earlier that your improvements to the file manager added a lot of much needed polish.

> but I did not have the skills or attention span to do it on my own.

This is how I feel about mos things now

KnightOS is a system I've been wanting to try for years, although I don't have the hardware for it. It's an admirable achievement.

How hard would it be to get KnightOS to be able to self-host? Something like an 8080 or Z80 is probably the wimpiest processor that it makes sense to try to build a self-hosted development environment on; although there were compilers that could run on the PDP-8 (address space: 4096 12-bit words), John Cowan tells me that in practice people did most software development for the -8 on bigger machines like a PDP-10.

CP/Mish is the other free-software operating system for the Z80 (and also runs on the 8080), but it's pretty far from self-hosting. As I mentioned in Dercuano, there's a free-software C compiler for CP/M which can run under CP/Mish (BDS C) but its dialect of C is not the one CP/Mish is written in.

A really small bootstrapping environment like this could go a long way to making computers comprehensible and making the Karger–Thompson attack infeasible.

Last time I checked, the source code can't even fit on the calculator's Flash chip.

Really? If not, it's not that far. I mean KnightOS/kernel/src is less than half a meg, and it gzips to 79K. corelib/src gzips to 10K, castle/*.asm to 6K, and all the core .asm files I could find to 107K. (I'm not sure I have the whole source because it's not in a single repo, so I may be missing something, and I've left out extra/, but still this is pretty far from four megs.)

4M is the best case, and that calculator is probably my least favorite because the screen is awful and the calculator can't drive it efficiently. The best case for a good calculator is 2M, but the most popular is just 512K. Keep in mind that you have to keep at least some of this file in RAM, which is 32K minus the size of your assembler itself.

I was considering the size of the whole toolchain, though. Perhaps if a different toolchain were made from scratch for this purpose... it might be possible.

Looks like the assembler has free-form syntax, simple text-replacement macros, assembly-time infix integer arithmetic, decimal, binary, hex, labels (including backpatching of arithmetic results, naturally), ifdefs, and some of the Z80 extensions to the 8080 instruction set? No stacks, recursive macros, conditionals in macros, typechecking, variadic macros, or user-defined opcodes? Roughly speaking it looks similar in power to the assemblers we used to use under CP/M, unless I'm missing something.

Yeah, but the CP/M had floppy disks and a more managable address space :) give it a shot, it could work!

Aside: at one point I was thinking about trying to port CP/M to these calcs...

Floppy disks are terrible. In NOR you can clear a single bit, and how long does that take? The least you can write to a floppy is a whole sector, normally 512 bytes, and the seek time is close to a million microseconds, although the actual sector write time is... well, maybe less than a hundred thousand microseconds. And they make a racket when you do it, too. And write endurance? Floppy disks have write endurance of about 1000 cycles.

Their only benefit is that you can take one out and put a different one in. But that means the software can't use a write-back cache and can't start writing to the disk except in response to an explicit user action that the user expects to initiate a write; otherwise you run the risk that the disk drive will be opened in the middle of the write.

Anyway, there were assemblers that you could fit on a single-sided, single-density, soft-sectored 5¼" disk along with your source code, object file, and "operating system", all in 90K. But things became a lot more practical if you could move up to a dual-drive, double-density system, so you could access two 180K disks at a time.

If there was some way to run a reasonable subset of CP/M programs on them, even if it required some hacking of the programs, that would be a significant benefit. I need to think about the problem more.

> But that means the software can't use a write-back cache and can't start writing to the disk except in response to an explicit user action that the user expects to initiate a write; otherwise you run the risk that the disk drive will be opened in the middle of the write.

You can just require the user to `umount` the disk before physically taking it out. And if a disk does get taken out before the read/write even starts, the OS can detect this and show a message `You MUST reinsert media $LABEL into drive: Abort/Retry/Ignore?`

I did use an operating system that required you to dismount floppies before taking them out (HDOS) but no popular OS or monitor program had such a requirement. HDOS was targeted at hobbyists and may have had some design features strongly selected against by the market.

> In fact, the “userspace” task switching GUI would read the kernel’s process table directly to make a list of running programs.

This is great. A perfect learning project is trying to build something yourself, then learning how other developers solved the problems or larger OSes handle it (or working backwards from that point and simplifying).

I noticed some "#ifdef FLASH4MB" in the linked source code, was it difficult to update it for 4mb or were similar constraints in place where it was basically the same with more memory?

Nah, it was reasonably easy. The main issue with 4MB calcs is that it used a secondary hardware port to add two extra bits to the page address. Until then we were able to page with only one port, so everywhere was hardcoded to use it. We had to replace all of that with a marcro, and update a few places manually. There's a comment in that code which speaks to this:

    ; we can't use getBankA because it behaves differently on the 84+CSE, and we don't want that

getBankA was the name of the macro we used to interrogate which Flash page was swapped into bank A (address space 0x4000-0x7FFF).

The real pain with that port was the new color screen.

Interestingly, "read the kernel’s process table directly to make a list of running programs" is exactly how the "ps" command worked in the distant past. It's a bit hard to find information about this nowadays, but I could find a couple of SE answers (https://unix.stackexchange.com/a/340893 and https://unix.stackexchange.com/a/511530) which mention this fact.

Relevant: Ndless[1], a jailbreak for TI-Nspire calculators (Nspire {CX ,}{CAS,}). I've played around with the source code[2] as well as compiling Rust[3], Nim[4], and Zig[5] to target it. It's all pretty neat, there's a gba emulator for the TI-Nspire floating around somewhere as well.

[1]: https://ndless.me

[2]: https://github.com/ndless-nspire/Ndless

[3]: https://github.com/coolreader18/rsspire

[4]: https://github.com/coolreader18/Nim/tree/tinspire

[5]: https://github.com/coolreader18/zigspire

Obstacles and difficulty make us better at things. Lack of obstacles makes us complacent. This is a known phenomenon.

https://www.ted.com/talks/tim_harford_how_frustration_can_ma...

Also interesting: PunixOS, a sadly-dead project to port 4.4BSD to Motorola 68k calculators like the TI-89. Not just a Unix-alike, but a Real Unix (tm), with pre-emptive multitasking and everything.

http://punix-os.blogspot.com/

Can you email hn@ycombinator.com so we can send you an invite to post that? Invited posts go into in the second-chance pool (described at https://news.ycombinator.com/item?id=11662380).

interesting, seems similar to fuzix http://www.fuzix.org/

I was about to post about the TI-92+ not getting any love, saw this, got happy, then saw it dead...sad again. Oh well. :)

You're only a Real Unix (tm) if you pay for a license.

As a former small-time contributor, KnightOS was an exciting dream. From the programmer's perspective, it got a lot of things right and really modernized the 6MHz environment from feeling like something designed in the 80s to something you felt like you could build a real piece of software on. I had written a computer algebra system for the 84 Plus and I was considering porting it to KnightOS because it would have gotten rid of a lot of baggage I had to do to support TI's operating system.

KnightOS never had the momentum it really deserved, and the calculator hacking community was far more involved with equally impressive projects in other genres (for example, the Axe Parser was an onboard compiler for a high level language designed for writing games on the 84 Plus). Still cool to go back and play with the website.

Neat. Would be interested to see a quick video/GIF tour.

Here you are:

https://yukari.sr.ht/knightos.webm

What's yukari.sr.ht? Personal file dump?

Aye. It has a sr.ht subdomain for historical reasons, but it's a personal server.

There goes my day.

Also, Lunix (C64)

At first I confused it with Temple OS. That was also very interesting but maybe less useful

honestly impressive, but yeah not very useful.

With no MMU or hardware supervisor mode, this would be more of a simple task executor than an OS in any reasonably-modern sense. Still impressive though.

Reply on news.ycombinator.com