Impending kOS (2014)
archive.vector.org.uk
archive.vector.org.uk
This is what I always strive for. Although I've only met Arthur a couple of times, he has been a huge influence on me as a developer. Besides some shared personal interests, his development philosophy that simplicity and consistency produces performance and malleability (to be able to constant add, subtract, or write).
He's on the top 3 developers I try to model myself after. While he's a god among men, his products (A+, K, KDB, Q, and now Sakti) make others mortal also gods among men by just learning to think and use his tools. My co-workers I'm sure get annoyed at my code-reviews ("delete this", "rthisewrite fir first with that then delete the second", "delete all this and return this",. ...) where I consistenly call for deleting a remodeling code to be as minimal as needed. I'm brutal about it - as I learn from K and it is partially Arthur's fault (and another dev I look up to). Maybe now they know where I picked part of it up.
https://shakti.com/ is his new distributed database project that is still somewhat hush hush, (and he not the kind of person that plays buzzword bingo - so I assume he has something special planned). I cannot wait to see how that turns out. I don't know if he's ever produced a bad product.
I'd be annoyed at this code review comment. "rewrite this `for` loop first, then delete the second `for` loop" is what I assume you intended.
The reviewer who gets to read the code gets to walk the shortcut to the solution after the trail was blazed for them: they get to think purely about the working solution without the intermediate steps. And they naturally will find places where the original developer was thinking in terms of his failed earlier attempts.
I bet that if I gave you a non-trivial problem to solve in a reasonable amount of time I could do the same kind of review on your code, forcing you to shorten it. And I will find something. Guaranteed. That is just how these iterations work.
Why are you framing this as though it's adversarial and the parent comment is stating some kind of superiority, and you're here to turn it around and put them in their place?
Isn't teamwork all about doing better than one person working alone?
In fact I am in the middle of fixing up code that I wrote as novel research because the powers that be want it in a production system. The code review I got was detailed and - frankly - embarassing at times because all the uncertainties and meanderings of research had me change pieces of code over and over and over again to a point where remnants of old approaches didn't stand out to me anymore and where I had at times built on top of bad decisions because they were meant as a temporary hack and experiment, but worked well enough and stayed in. And by the time I submitted the code for review I had become blind to these mistakes. I know how to write good and clean code and I get through code reviews easily when I know what I am doing. Not this time, though.
Kdb is fantastic as an app server framework. The q language is just enough syntax help on top of cryptic apl to make it useful to a mass audience.
We need a free and open project that replicates this functionality.
It is interesting how the closed source issue is consistently raised on HN with respect to k when there are numerous other examples of closed source software used by HN readers where I never see this issue raised. In those cases it does not seem to be important. For some reason however, it seems availablity of source code is important to some folks in the case of k.
I am curious; What would you do with the source code? Is it an issue of verifying what the software is doing, or perhaps being able to modify the software or make "fixes"? Is it something else? Maybe you simply want a free option for commercial use. If the later, please disregard my curiosity.
For me the issue I have with k being closed source has always been ports, i.e., limited OS support. They used to have a FreeBSD port many years ago, but today there is no support for OS other than Windows/Solaris/Linux, e.g., NetBSD.
The guys who license kdb/q probably make more money today from service/consulting agreements and not the software licenses. So it's to their benefit to go open (a la MongoDB)
Also, if you're building a tech stack from scratch - you want it to be "clean". You don't want to muddy it with expensive/onerous software licenses.
It's a public company, so you can go check:
https://www.firstderivatives.com/wp-content/uploads/2019/06/...
Software earns more than consulting, and the profit is better too.
Just because Mongo isn't good enough to make money on its own merits doesn't mean anything.
> Also, if you're building a tech stack from scratch - you want it to be "clean". You don't want to muddy it with expensive software licenses.
I've been in probably a hundred funding and acquisition meetings at this point in my life, and this has literally never come up.
/emul/linux is how I use k on NetBSD. However I hesitate to rely on Linux emulation because it does not seem to be something that is important to current BSD developers. If I am not mistaken, both OpenBSD and Dragonfly have removed it.
This might not have been feasible or practical a few years ago.
Office productivity applications have a lot looser latency requirements, and tend to change the screen less frequently than games too. But current solutions are capable of playing videos and some gaming.
Unfortunately, there's only one Arthur Whitney. Other people can emulate his style and genius, but I don't know anybody else who brings this kind of ruthless minimalism to everything they do -- and make it work so well.
And check suckless.org, especially dwm
Maybe if we all thought like Arthur we'd just use gopher and be able to write a new client or server from scratch in an afternoon.
To be sure, software does have some bloat. But some of the complexity is necessary for software that can be used by the whole world.
Imagine starting from scratch with Genode or what Google is doing with their new (3rd) OS.
I agree that's important, but I don't think that a great OS should become just a good OS in pursuit of that goal. After the OS is made, it seems like there would absolutely be ways to make the system accessible as-needed.
Really? All? And if not all, are we back to the ADA reasonable accommodation standard? What bugs me about the "accessible or bust" people is that they will deprive 99% of the population of something because 1% can't have it. There's some on-going case about university courses in California of some sort being publicly released but "not being accessible". So now, no one can access it because they got taken to court. It costs zero dollars (or very few) to publish existing material to the world; it costs quite a bit more to add stuff to it.
Maybe a great OS should be available which can run quickly without accessibility features, but slower if they are explicitly installed by an end user. Like it or not, it's a balancing act between how much we're willing to give up and for how few people.
Most of the standard conviences available to "normal" people because smartphones are a thing and yadda is how I make my life work.
Among other things, I also co-own a tiny list for blind developers. So I have some familiarity with the fact that there are people who actually need accessible design in a way that I don't.
But I think any internet that works well dramatically improves life for the vast majority of the 60 percent of people who have varying degrees of impairment, two-thirds of whom do not self identify as disabled or handicapped.
Cars were invented for people with sufficient use of their limbs and eyesight. After their invention, the control systems were adapted for people who, for instance, can't use their legs. I don't know of any way they have been adapted for e.g. blind people. And yet, even though they are not accessible to a large fraction of disabled people, the world is better off with cars, designed for not-disabled people and then adapted as far as possible, than without cars.
That's all I'm getting at. Its too much of a constraint on invention, but likely to be adaptable later.
B. If it wasn't clear, I'm generally on your side here.
However, I'm not convinced that our OSs can't be much, much simpler. You can, almost reasonably, use DOS as a daily driver still. I have very fond memories of DOS because it was simple. Even as late as the early 2000s, if you wanted something like a MAME cabinet you'd often use DOS. You just plop some application and data folders onto a disk with like 4 system files and you were pretty much done.
Where is today's DOS? I suppose it's Linux, which is orders of magnitude more complicated (especially once you start talking about a GUI) and even Linus admits it has become quite bloated. I personally think we can do a lot better and I've been toying with the idea of putting something together because I'm just so sick of modern computing's bullshit.
PS: If anyone is aware of an organized effort towards this goal that already exists, please let me know.
Still, AppImage is an attempt to bring sanity to the garbage fire of the Linux Desktop. I'd rather have a system that wasn't a garbage fire to begin with.
You seriously say this in an era when Electron exists? You'd be hard pressed to waste more resources by simply not having deduped libraries.
> Package-based distributions concern themselves with this a lot
Yes, and they end up causing a lot more headache than they ever save because of it.
If you want simple nowadays, boot a Linux kernel into busybox. It'll boot almost as fast as DOS but you'll have all the crazy modern amenities like 64 bit support and network drivers.
You don't even need a package manager. You could just unpack a GCC binary and compile all the junk you want yourself.
I like your rule-of-thumb that you should not need a package manager.
Muratori has a theory that USB has made the OS scene complex (https://www.youtube.com/watch?v=kZRE7HIO3vk&t=1350s).
Two other things - the browser and TCP/IP.
Goal: a system where you can fit the whole stack in your head, yet participate in a networked world.
Approach A. Outline a reference hardware platform. Port NetBSD or minix3 or plan9. Then simplify. e.g if unix, get rid of users and groups, get rid of x, get rid of package mgmt, get rid of nfs. You could bootstrap this in qemu.
Approach B. Establish an alternate browser. Something like gopher, but with async events between the 'page' and the user. This allows for chat and form manipulation. You can send non textual content through this async link - video, audio. Codecs are a trap, they lead to pkg management and complexity. How to stop proliferation?
Linux above: You need a hypervisor. See L4 or Genode.
Linux below: See unikernels.
An alternative would be targeting a single set of hardware that's likely to be around a really long time. I'd be ok with that and something like a Raspberry Pi, but only if all the hardware was open. Of course, then you're in more of a 68k Mac space where hardware choices are extremely limited.
I think we are talking about a different niche.
If so, no need to think in terms of catch-up. Rather: work out the design values of this niche.
Lots of ideas, but this thread is already deep. I have created #dinghy on irc.freenode.org, would be interested to discuss further.
2. about web: too bad that design-addict webdev community bloated it. But wish someone make a simpler, faster & straightforward web as portable app platform with few nice commercial qualities...
3. About OS: OS need not be bloated if it isn't running behind popularity by satisfying everyone's remote needs. ( source: plan9, suckless, busybox )...
2. No disagreement here.
3. Unfortunately it still kinda does, because drivers. Two of the ones you listed sit on top of the Linux kernel, which even Linus thinks has become bloated.
Unfortunately the only way to deal with that is to give up on the idea of running on a majority of existing PC hardware and target only a small subset of commodity hardware. Ideally it would all be open, to ensure it isn't going anywhere for a while, but sadly there is no such system.
I'm glad to see I'm not the only one who thinks Users and Groups are the wrong abstraction for systems like this.
\l view.k
/key(return back ^delete)
kx:kr:{u,:,(j,j+#x;*k_a);e[k]x};kb:{$[=/k;J j-1 0;];kx""};cd:{J j+!2;kx""}
/edit(undo cut paste)
e:{a::?[a;x;y];J(*x)+#y};cz:{$[#u;e/_`u;]};cx:{kx cc`};cv:{kx@9'`}
/save
cs:{n::$[#n;n;0'""]1:a;r::a}
It's not something I find appealing, personally. Then again I am a python guy, which is about as far from this as you can get.
Is he saying k is a lisp? I'm sure there are important differences here, but I don't know enough to say what they are. (There is a question implied there.)
The talk is by the fourth member, Geo, who also frequents HN (geocar).
the new project. Arthur has a tendency to turn mid stream sometimes. I'm sure much of that code was used in this new DB he's working on (db's and os's are quite similar).
https://gist.github.com/chrispsn/da00835bb122c42f429a084df83...
Also, there are plenty of memory allocators to choose from, and every runtime that doesn't use the libc has to ship its own (if it does memory management, of course). The OS only gives you big chunks to use as you please, and you can't sidestep the syscall. So that's not a super novel thought.
It's very optimized for in memory use, so assuming the db is all in working memory and the code operating on the data is in L1/L2, and for the type of data in the column store (Kdb+), you usually pull related chunks into the Lx data cache on which you filter. I would say k is fast for the use case where column memory based storage is a good fit, so relatively small data sets that live in memory and where the operations on the data fit column stores well; I would bet almost all k programmers (including myself) mostly have experience with those datasets or have shoehorned less suitable cases into that case because it did not perform otherwise.
There is obviously no magic, besides, for many 'new' programmers and for current standards, the incredibly small binaries for so much functionality. And that helps with always staying in cache vs almost every other software you will use these days where there is continues chunks of instruction data going to and from cache. That's why C (and maybe Rust some time in the future but the binaries were too large when I last checked) is still important if you really want to get to that point.
Ofcourse in that 200something kb k/kdb+/q binary there is no room for much optimizing, that is why it absolutely falls down for cases it was not optimized for. And when you use a lot of k you know how (and automatically do) to shoehorn basically anything in those cases.
It is not magic by any means, but it comes closer to magic for many as it is alien like assembly compared to JS or C# or something like that I guess. And the speed comes from keeping to your niches.
> Also, there are plenty of memory allocators
As I understand it, he did it because he did not find the Windows memory allocator efficient enough specifically; not sure why he didn't check for other allocators; I guess because he was optimizing a runtime, he just rolled his own instead.