Why do we need a new OS?
3lproject.org
3lproject.org
Nonsense! Why would you waste any resources on infrastructure when you could use them for applications. Every cycle that your garbage collector runs is a cycle that the cpu could have been sleeping and saving battery life. This concept that we don't need to care about resources anymore is absolute poppycock.
For the same reason we waste them on application frameworks instead of more interesting computations - because applications are written by humans, who take a long time to do it and are notoriously bad at it. Every cycle your garbage collector runs supports the existence of software that might not exist yet, or ever, without it.
It isn't that simple. Depending on your program it can actually be slower to do manual memory management. malloc isn't free. In some cases allocating memory and performing GC is actually faster.
It's not that we don't have to care about resources at all just that we can sacrifice some of the resources to improve the situation for users and programmers. If that isn't your thing then feel free to write everything in assembly.
This is false. The reason your phone has the battery life it does is because the OS and libraries and applications have been written to allow the hardware of the phone to drop in to lower power states as much as possible. Unless the language you have written your OS in has some bizarre requirement to wake up all the hardware constantly then the language doesn't matter a ton when it comes to battery life. What matters is writing programs that wakeup the hardware as infrequently as possible.
> we can sacrifice some of the resources
Which one is it?
look at pypy for example: Python interpreter written in Python that's faster in many cases than cpython because they could build advanced features for optimising the underlying running code. writing an OS in C because efficiency seems like a load of hooey to me unless you have facts to back up your idea that for some reason the OS has to be the efficient thing.
software development comes down to 1 thing: delivering features. who cares if you need twice the servers to run your product of the alternative is you run out of runway and your project gets canned. hardware is cheap, engineering time is not
I couldn't help but wince at this. I feel like this isn't a healthy attitude with respect to programming.
Ask yourself if the service you're trying to create would have been valuable, say, 4 years ago. Yes?
Okay, suppose you wrote it in C++ 4 years ago on four year old hardware? Is it still valuable? Yes?
Okay, now write it today in Node and run it on modern hardware. The hardware cost is the same, the service exists, and it was presumably cheaper to write and maintain in Node. Is it still valuable? Yes?
That's why it's reasonable to write apps in Node.js today: the performance and cost are the same as doing it in some "more efficient" language 4 years ago, except the cost to actually build the thing now in Node is (much?) lower.
If you were okay with the hardware costs for the service 4 years ago written in C++, you should be okay with the hardware costs for the service today written in Node.
You can use this argument to support "slow, inefficient" languages for many different kinds of projects (though not all, obviously). And I'm an efficiency nut.
- OS's were designed for limited resources back in time. Now it's time to design something for unlimited resources.
- Attempting to retrofit a new security model to a current OS would require everything would be rewritten so we decided to rewrite an OS, which will require everything to be rewritten for it, including the OS this time.
- Sandboxing, local-scope FS access, user controlled permissions have never been done before, so we're doing those.
- Runtime analysis will give out better performance
- Compilers have always been hindered by OS design, so our new OS will break the barriers of compilers (finally! )
(and more!)
Do I, as a user of the OS gain anything, or is it just easier for the kernel developers to not have to care about that?
Not to mention we are bound by Battery technology. In that sense, we are still very limited in resources.
Regarding batteries, the real issue there is how often you wake up the hardware from low power states. An OS like 3L could actually control that better than what we have today and give us better battery life.
Even if you don't gain anything directly you still gain significantly indirectly. Improvements in the speed and quality of kernel development means end users see new features quicker and bugs fixed sooner. It also makes it easier for CS researchers to try things out and make large advancements that can be easily applied to the OS.
When is the last time you needed a feature that could only be provided by your kernel?
> It also makes it easier for CS researchers to try things out
How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)? Your premise is that life will be better because we won't have to deal with the ugliness of talking directly to hardware in C/C++/assembler, so what room does that leave for research?
Also I just want to tag on a question based on one of your reddit posts:
> One example is that 3L can save the entire system as you are using it. This means that if the power gets cut or the battery dies you can (instantly) reboot the system and it will be like you never lost power.
You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
Generally these are "unseen" features such as adding multicore support or adding support for a power saving feature on your graphics card.
> How are they going to try things out if your OS doesn't support any of the techniques they're interested in (i.e. novel ways of getting at hardware, novel ways of doing virtual memory, etc)
They are interested in a lot more than that. Check the 3L references page for a few examples. [0] Plus you can do many of those things with 3L.
> You realize that this sort of thing is a huge area of current research, right? Because it's a really, really hard problem?
Yes. But it is certainly possible and has already been done to some degree. Genera, for example, did it and a number of other real-world and research OSes have done it. Of course the real challenge is making it as real-time as possible but getting it most of the way there isn't actually that difficult.
http://www.symbolics-dks.com/Genera-why-1.htm
About time we have something modern that can do everything in that ad that's relevant today. I want my live-updating, live-fixing, per-function compiling, whole-system debugging, crash-resistant OS. If they could do it in the 80's then... you know... ;)
Genera (and other lisp OSes of the time) proved a lot of what 3L is building on. Unfortunately at the time the hardware required to run genera was expensive. Today the atmosphere is different.
No. Almost any language we use today can run on 3L and transpilers are planned for the future of 3L. The security features will still be provided via the underlying lisp though. Technically the same security mechanism could work for most languages but it is much simpler, easier, and more natural to implement in lisp.
Even languages like C can be supported safely via memory emulation (although right now that would mean it runs quite slowly but likely, although no guarantee, it will run well enough in the future as hardware becomes more powerful).
Old stuff is bad except when we like it, I guess.
[1] And quite dishonest to lump WinNT in with that...
So, yes they suck. Yes superior architectures or languages of the time are worth copying or adopting over the knockoffs created due to hardware deficiencies. This is often even provable where one can look at problems that crop up in technology X caused by design point Y that's prevented by alternative design Z. And one of those alternatives was from 1961. ;)
Lisps have also changed since it was first created. Unlike many languages lisp has evolved tremendously over the years and has benefited from a tons of language research. The lisps we have today are much more than we had in the 60s.
The advantage is it makes development of the OS easier and application debugging easier. It is also makes future proofing the OS more robust. A program that can rewrite or inspect other programs is very powerful, in the same way lisp macros are.
One concrete advantage of this is you could write a debugger as a user application, it wouldn't have to be integrated in to the language runtime.
And is a security nightmare. If I have an executeable, I have verified, I don't want the instructions to be altered in any way. In fact I prefer if it is impossible from a hardware level by marking them as executeable bits.
Also, 3L runs in two modes: development or secure. In secure mode you can completely disable manipulation of source code (or strip it out) and mark all code as read only and have the system verify the code hasn't been modified when it is run.
You still need a debugging build, otherwise compiler optimizations will obscure the executeable to the point where debuggers cannot reliably debug it.
If your environment doesn't support C, how will you ever find code to run on it? Where is your GNU project?
Technically it could support C. One of the reasons it is based on Scheme though is that there is a lot of code already, code written in Scheme. :)
Example from sklogic that embeds ML in LISP https://github.com/combinatorylogic/mbase
Take a look at my own Haskell operating system project Hos (https://github.com/tathougies/hos). Written in Haskell 98 using the JHC compiler (a bit outdated, but makes it easy to generate ANSI C from Haskell sources). Currently supports multi-tasking, interrupts, paging, garbage collection. Has the beginnings of a simple user-space ATA driver.
Patches welcome. E-mail me at travis@athougies.net if you want help setting up a development environment!
An example implementation from CHICKEN Scheme: http://wiki.call-cc.org/man/4/Types
Some of that is vastly simpler to do with a dynamic language.
"You can boot into a REPL (Read Evaluate Print Loop) and can inspect or rewrite any part of the OS as you are using it."
As far as I know, nobody has figured out a way to have your cake and eat it too with static typing. Some of the simple things that lisps have been doing for decades are very difficult if not impossible to do with statically compiled languages.
But it's quite a lot of work to get acceptable OS performance if your entire system is interpreted or has to be JIT compiled all the time. People who do that tend to coincidentally decide they're so hardcore they don't even need JPEG support, video, or web browsing…
But I would absolutely love to be proven wrong...
I do not know what's referred to as "original OSes" but many old operating systems had actually pretty decent security architecture. MULTICS and VMS, for example, and many others too.
I'm glad that alongside the philosophy of this being a FOSS endeavor, there's the practical aspect of this being adjustable on the fly by users in the old Lisp machine tradition.
Though I'm sure it's too early for outsiders to shape the PoC code, is it available anywhere for us all to follow along?
Is there the possibility that the OS code will be so mutable that it is impossible to predict if an application will run on any one installation? Will there be any posix-like standard to guarantee compatibility?
A mailing list to follow development will be available soon unfortunately I haven't had time to get it up and running yet.
edit: but check back to 3lproject.org soon or the kickstarter page. If you have kickstarted it I'll also make sure to send out an update.
In summary, 3L drops explicit support for filesystems. Instead the system just provides a mapping of labels to storage blocks. From there those blocks can be utilized in any fashion you like. A filesystem library can be written to manage its set of blocks as it pleases. A database can skip the overhead of a filesystem and directly access the blocks to improve performance.
But yes, assuming you have imported the filesystem library (which has probably already been done assuming you are at the REPL) then you can process files as regular lists of lisp data (or in whatever language you want really). Not sure that is really that special though since you can already do that.
http://3lproject.org/references
I understand than in short, the security model is about how to make sure a process does not do more than what it's supposed to. In this case it's lisp environments that provide the necessary sandboxing.
Privileges for a game:
- Can use full screen.
- Can read own content installed with app.
- Can read/write persistent storage associated with app.
- Can communicate with domain "thisgame.ea.com".
- Cannot run in background, access address book, ...
Word processor:
- Can access files mentioned in user file dialogs (file dialogs being run from a separate process).
- Can transmit requests to send mail to app "mail".
For any app which accesses information outside its own content and own persistent storage, the user should have the options "allow", "deny", and "fake". If a program asks for address book access and won't run without it, you can give it the "fake" option, which gives it access to a dummy address book. (I've seen something like this demoed for jailbroken Android phones.)
The amount of data XPrivacy [1] can fake is impressive. It's a shame it isn't a built-in feature.
On top of that the implementations have all been regularly updated over the years and many are extremely mature and well-optimized.
Do you equate newness with quality, as seems to be common on HN?
Perhaps an OS written entirely in this month's JavaScript framework, deployed on Docker, using monads-as-a-service or whatever the hipsters have come up with lately would be more your cup of tea.
When the link's thesis is "these OSes design years ago are old and therefore bad" it is hardly unreasonable to point out the language the author prefers is even older.
Whereas already in the 60s, we had hardware systems like the Burroughs b5000 that could be fully programmed in Extended Algol and Espol, with zero Assembly.
The idea, as I see it, to base modern OSes in better older architectures that were lost to the mists of time.
Actually the mobile OSes, with their memory safe languages seem to already be moving into this direction.
That's not the link's thesis. Equally flippant but more correct would be to say "these OSes were written in memory-unsafe languages because hardware limitations demanded it back then, therefore they are bad"
Security Kernel https://www.kyleisom.net/downloads/security_kernel.pdf
Formally-verified Scheme http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=881...
The features could certainly be worth it. Some aspects of the Genera experience, esp productivity & holistic debugging, still aren't in modern flows:
http://www.symbolics-dks.com/Genera-why-1.htm
So, who knows where it will go. Betting on a semi-functional language with prior high-security and high-productivity work is a better start than most projects like this, though.