HNHacker News
TopNewBestAskShowJobs

nderjung

545 karma · joined November 16, 2018

submissionscomments
nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
It really depends on your usecase here. If the many application processes rely on the same OS libraries, versioned language runtimes (e.g. same python version), kernel version, etc. AND you trust the OS then it may make sense. However, unikernels offer the lightweightness of a container process with the security of a VM. In addition to this, memory ballooning[0][1] and other resource-elastic features are available to VMs too: allowng you to under-provision them and then later increase resources when load demands it.

Unikraft unikernels can also be managed using the same orchestration tools as containers, check out the talk at CNCF[2].

[0]: https://github.com/unikraft/unikraft/pull/219

[1]: https://pmhahn.github.io/virtio-balloon/

[2]: https://www.youtube.com/watch?v=cV-xawN9_cg

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
It works both ways with Unikraft, either bring an existing application and let it run with the added performance/security (and think it's on Linux) or write your application with our performance-oriented APIs[0][1].

[0]: https://unikraft.org/docs/concepts/architecture/

[1]: https://usoc21.unikraft.org/docs/sessions/10-high-performanc...

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
We're working hard on adding SMP support to Unikraft which is planned for the next release. The PR you have linked has all the details about the on-going work!
nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
The application code is compiled along with kernel code so you can think of unikernels as a single-process VM, there's nothing else running other than the application so it boots straight to `main()`. Unikraft just facilitates the runtime of the application to be able to run as a VM or on baremetal. There's no shell, so you can't instantiate another program from disk. If the application wishes to read and write from an attached storage, it can, but it can't start another process if it's reading application code to-be-executed. Starting another process is a bit tricker since there is no `fork()` to execute another process. Interesting work is being done to enable multi-threading across cores via SMP[0] however and to provide fork like ability but with regard to the application's logic[1] but not in a wider general multi-processing environment. I hope this clarifies things.

[0]: https://github.com/unikraft/unikraft/pull/244

[1]: https://xen2021.sched.com/event/jAME/cloning-unikernels-on-x...

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
1. We build unikernels using the 'kraft container' which is Docker/OCI image[0][1] which has the necessary build tools to build Unikraft unikernels. We plug this into Concourse CI which builds thousands of combinations of Unikernels[2] as part of our code review process[3]. In addition to this, we have on-going research and tooling to help automatically discover permutations of Unikernel builds[4]. We then run the unikernel natively using the target platform/hardware combination or use QEMU emulation.

2. Really great question, but mostly you can expect the same functionality of an application when it runs as a unikernel because the application "thinks" it's still running in a traditional OS environment -- as it should be. Check out this documentation[5] (after step 7) about porting, it has snippets about where the boundary sometimes breaks. Even then, you can run it in POSIX-compatibility mode[6].

3. Well, general-purposes are not suited for deployment environments (which is what Unikraft is suited for). Installing Gentoo (or Ubuntu, Debian, for that matter) is a waste of resources if you only SSH in once to install your desired application.

[0]: https://unikraft.org/docs/usage/install/#docker

[1]: https://github.com/unikraft/kraft/tree/staging/package/docke...

[2]: https://builds.unikraft.io

[3]: https://unikraft.org/docs/contributing/review-process/#stage...

[4]: https://github.com/lancs-net/wayfinder

[5]: https://unikraft.org/docs/develop/porting/#providing-build-f...

[6]: https://unikraft.org/docs/features/posix-compatibility/

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
Thanks for the feedback, we're in the process of adding a security section[0] which will detail more on the on-goings, but we'll work on adding more highlights on the main page.

I need to highlight we have separate research[1][2] which will make its way upstream soon which aims to provide hardening between internal libraries (e.g. isolating the network stack or scheduler) using gates like Intel MPK or separate hardware-accelerated services.

[0]: https://github.com/unikraft/docs/pull/32

[1]: https://project-flexos.github.io/

[2]: https://github.com/project-flexos/unikraft

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
It's possible to run Unikraft unikernels via kubernetes with minimal interference to the ecosystem, check out the talk at CNCF: https://www.youtube.com/watch?v=cV-xawN9_cg

We'll be launching managed kubernetes support for Unikraft unikernels soon too at https://unikraft.io

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
Unikraft is based on a small trusted compute base, meaning there is nothing else running with a unikernel, no ssh, no daemons, no Linux, etc.

Towards increasing security, however, we have just introduced native support for Rust[0] in Unikraft, paving the way for more internal libraries to be based on this secure and performant language.

[0]: https://github.com/unikraft/unikraft/pull/348

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
It's possible to create an IPSec + firewall based on the Click Modular Router[0] and run this on top of Unikraft[1].

[0]: https://github.com/kohler/click/wiki/IPsecEncap (and other IPSec* elements)

[1]: https://github.com/unikraft/app-click

It could make for an interesting tutorial with a full Click-based IPSec router though! :)

nderjung··on Unikraft is a fast, secure and open-source Unikernel Development Kit
Hi @edsiper2, if you are running into any problems I'm happy to help, we can chat directly on the Unikraft Discord server: https://bit.ly/UnikraftDiscord
nderjung··on The battle of the computing clouds is intensifying
This is true, but the same principle applies to EC2 instance. We also tried renting an m3.medium and installed Debian+NGINX on one and Unikraft+NGINX on another and experienced the same cost savings with, in addition, twice the performance (roughly 12K req/s vs. 26K req/s).
nderjung··on The battle of the computing clouds is intensifying
Whoops :) Yes I am affiliated, I can't update my comment but I'll make sure to indicate this if/when I write a comment which discusses anything about Unikraft.
nderjung··on The battle of the computing clouds is intensifying
It's really cool work the folks at CloudFlare are working on. The trade-off there though is that it's still a process isolated by the OS and so relies on OS-level security via the use of syscalls. Even if context-switching is reduced with Deno, making privileged calls to, say for instance `open()` or `socket()`, still impose wasted CPU cycles as the OS checks whether the process is allowed to perform this action. With unikernels, the call is allowed (and no extra CPU cycles / instructions are called) since the unikernel is its own VM and is isolated by physical hardware separation.
nderjung··on The battle of the computing clouds is intensifying
There are many ways to debug unikernels in production, particularly, we have in-kernel monitoring (e.g. ability to enable fine-grain memory, scheduling, and resource monitors. These work also any included libraries). You can also produce flame-graphs of the entire unikernel image which show the entire kernel stack and the application. There is also remote GDB support (which come with additional scripting tools for automation) for debug-images which are built along-side the production image.
nderjung··on The battle of the computing clouds is intensifying
We are launching a pricing page in the next month as we are gearing up for launch :)

One example of recieving a 50% cost saving is from Amazon Machine Images (AMIs). We compared the cheapest official NGINX AMI available on the Amazon Marketplace[0] and then also ran an NGINX+Unikraft AMI. We ran the same workload using wrk[1] and then checked the bill at the end of the month, roughly $80 vs $40.

[0]: https://aws.amazon.com/marketplace/pp/prodview-xogyq23b3mfge

[1]: https://github.com/wg/wrk

nderjung··on The battle of the computing clouds is intensifying
One way to reduce compute costs significantly is by using a much lighter stack. Typical deployments to the cloud are based on the traditional OS model, and more often than not are a container-in-a-VM solution to deploy some service. The reality here is that there is so much unused, unnecessary and wasted compute resources. Why deploy a full OS and kernel which will only ever serve static web content, for example. And then there's the attack surface that comes with a full OS, running additional services like SSH which may only ever be used once.

Major vendors like Amazon and Google are happy to provide this to anyone and with simple on-boarding, but, as is clear in the article, it's expensive. With Unikraft[0], you can completely re-package your application as a unikernel, automatically. It's a single VM image and does nothing more than just running the application. It boots straight to the main application, it doesn't have a shell, and has nothing else running along side it. The result is a huge cost saving (up to 50%), since you're no longer having to pay for these extra resources just to run the a full traditional OS image plus your application.

[0]: https://unikraft.io

nderjung··on The big idea around unikernels
Hey, Unikraft has support for running the Python[0] and Ruby[1] runtimes as unikernels. We're about to release PHP too :) Watch this space!

[1]: https://github.com/unikraft/app-python3 [1]: https://github.com/unikraft/app-ruby

nderjung··on Ask HN: How are you using unikernels?
[Unikraft maintainer]

Your question breaks down into two I think: 1). whether someone working on a typical "userspace" application exclusively and then decides to package the application as a unikernel right before deployment and 2). whether you are working exclusively on a unikernel-centric application, that is, you are making use of non-POSIX, internal (kernel) APIs in order to exploit better performance or better interact with your platform (e.g. Xen, KVM, etc.) or hardware architecture. Unikernels can achieve excellent performance as they have a very tight coupling with their surrounding environment.

In the case of 1), the developer workflow is the more or less the same in my opinion, you build your application in the comfortable way that you do before, without having to worry about the underlying kernel. There's two modes to this though. This is essentially whether your appplication is built using an interpreted language or a compiled one. In the former case, a python program is an apt example here, like using a framework like Flask where you build your HTTP APIs and work on application logic etc. In this case, you can use a pre-compiled unikernel which is tailored to run the python runtime. Unikraft makes these available actually[0]. :) When you are ready for deployment, you simply mount a filesystem with the source files of your application and point to the entry program as you would normally, like invoking the python3 binary with a path to your python program. When a developer is building some application which is compiled, e.g. with Golang, you can follow the same workflow as you would normally. However, when you are ready to deploy, you do something called binary rewriting. Here, you change invocations of kernel-space methods, mainly syscalls, to JMP instruction calls which are addressed inside of the unikernel binary. A final linking step puts the two together and you are then able to run your application as a unikernel. HermiTux does something like this[1] and Unikraft is soon to release some new tools to accomplish this too! Stay tuned! :) Unikraft aims to be POSIX compliant, and you simply select the relevant libraries and options you need to get the features your application needs to run. This way, you don't have to rely on maintaining two versions of your application.

In the case of 2). where you are building a unikernel-centric application, it's no more different than working with a compiled language. At least in my workflow, when I am working on some application or a new internal library for Unikrat, I just re-compile the source files I'm working on and then perform the final linking step to create the unikernel binary. I mainly use kraft[2] to help me acquire the relevant source files and libraries I want to use and work on. The kraft repo ships with a suite of docker images, and this for me is my main way of creating a dev environment where I have the right version of gcc, qemu, and all relevant tools for debugging too. I typically invoke this environment like so:

```

cd to/my/project/path

docker run -it --rm -e UK_KRAFT_GITHUB_TOKEN=<set token> v $(pwd):/usr/src/unikraft --device /dev/kvm --entrypoint bash unikraft/kraft:staging

# you are now inside a container

kraft list update

kraft init

# ..etc..

kraft configure # or kraft menuconfig

kraft build

kraft run # or use qemu-guest -k ./build/my_kernel

```

I hope this helps!

[0]: https://releases.unikraft.org/unikraft/apps/python3/stable/

[1]: https://dl.acm.org/doi/pdf/10.1145/3313808.3313817

[2]: https://github.com/unikraft/kraft

nderjung··on Unikraft – Fast, Specialized Unikernels
This does not mean you cannot still use pip to manage Python dependencies. Simply package all the required addditional libraries from pip through a virtualenv/venv, and package this in the filesystem that the python3 Unikraft unikernel reads on boot.

For more details, check out the library for details: https://github.com/unikraft/lib-python3

nderjung··on Unikraft – Fast, Specialized Unikernels
Additional links:

* Homepage: https://unikraft.org

* Docs: http://docs.unikraft.org

* GitHub: https://github.com/unikraft/unikraft

* Twitter: https://twitter.com/UnikraftSDK

* LinkedIn: https://linkedin.com/company/unikraft-sdk

nderjung··on Unikraft – Fast, Specialized Unikernels
Not quite. To get started building Unikraft unikernels, you can use kraft[0] tool which builds a bootable image. At the moment, this is the entrypoint into the ecosystem and allows you to build, for example, NGINX for KVM, Xen or bare-metal.

Having said this, we have on-going work to integrate Unikraft into Kubernetes. We are very excited about this work and will be making noise and providing extra details about this very soon :)

[0]: https://github.com/unikraft/kraft

nderjung··on Unikraft – Fast, Specialized Unikernels
Note that this is a VMM and not a unikernel project. We can create bindings with this VMM for faster boot times and access to fast IO paths.
nderjung··on Unikraft – Fast, Specialized Unikernels
I believe there is growing interest in providing leaner, "trimmed" runtimes for services deployed to the cloud. Today, this is seen largely by specializing the Linux kernel for, for example, container services[0] or in general[1], as much as that is possible (the paper above covers this problem in greater detail). But, Unikernels in themselves are not yet widely adopted. This is the space Unikraft is aiming to enter, providing the ultimate level of specialization for a target application.

It's clear that bigger players, such as Red Hat[2] are interested in the topic of unikernels, and that cloud providers are preparing for this future too [3].

[0]: https://github.com/linuxkit/linuxkit

[1]: https://github.com/hckuo/Lupine-Linux

[2]: https://dl.acm.org/doi/10.1145/3317550.3321445

[3]: https://firecracker-microvm.github.io

nderjung··on Unikraft: Posix-Like Unikernel
Hi, maintainer for Unikraft here. (:

Thanks for posting thhe links! We've done a few more talks, including FOSDEM this year[0] which we still need to add to the website. Thanks for the suggestion about the website, we're continiously looking for ways to improve. We're hoping to revamp the tutorial with new updates to kraft[1].

[0]: https://video.fosdem.org/2021/stands/unikraft/ [1]: https://github.com/unikraft/kraft

nderjung··on The RustyHermit Unikernel
Did not realise Visa was using z/TPF. Do you have a link with more information?
nderjung··on The RustyHermit Unikernel
10s of milliseconds (fig. 4)[0]

[0]: https://pub.nderjung.net/papers/emsoft20.pdf

nderjung··on Cut Your Cloud Computing Costs by Half with Unikraft
Link to the project on Github: https://github.com/unikraft/unikraft
nderjung··on OSv – Linux binary compatible unikernel for virtualized environments
Be sure to also check out Unikraft, a Unikernel SDK [0]!

[0]: https://github.com/unikraft/unikraft

nderjung··on Git is too hard
To answer your question about deleting a remote branch: branches are usually features, e.g. fixes, new features, etc. Once I've pushed to the remote and merged, the branch is no longer necessary. This is why someone would delete a remote branch.
nderjung··on Ask HN: Who is hiring? (December 2019)
Lancaster University | Chair in Computer Networking | Lancaster, UK | Onsite | Full-time | Salary minimum £66,285 + benefits | A2588-R[0]

The School of Computing and Communications is seeking a new Chair in Computer Networking to join our team. The appointment will be at professorial (full or tenured) level. We particularly welcome applications from strong and collegiate individuals with an excellent track record in areas of network research (including but not limited to): Software-defined Networking and Network Functions Virtualisation, Network aspects of IoT and Cyberphysical Systems (CPS), Network Security and Resilience, Quality of Service/Quality of Experience and Future Network Architectures.

Lancaster University has been a leader in Computer Networks research and practice for over 25 years. From original work on ATM, Quality of Service and Programmable Networks in the 1990’s, through multimedia systems, IPv6 and Wireless Mesh Networking in the 2000’s, to the very latest research on SDN, NFV and resilient networked systems – an underlying theme of our research is around experimental, testbed-based activities having real-world impact.

We invite applications from candidates with appropriate networking domain knowledge from academia, government and industry. The Chair must be able to address academic and external audiences, have an established international reputation for excellence and impact in their field; a proven ability to support and direct a team, and secure external research funding. You will help maintain and build excellent research, and education delivery; positioning the School as a leading source of expertise in the area of networking. We expect candidates to be committed educators and to contribute to the development of our educational portfolio, including our international partnerships.

The position will have the unique opportunity to contribute to the growth and future direction of our multidisciplinary research institutes at Lancaster in Cyber Security, Data Science, Social Futures, Material Science, and the new £41m Digital Health campus.

The School offers an open and collaborative environment that is both highly inclusive and stimulating for career development. The University aims to promote work-life balance for all employees and the School will consider a range of possible flexible working arrangements in order to recruit the best candidate. The University is a member of the Athena SWAN Charter, which recognises and celebrates good employment practice undertaken to address gender inequality in higher education and research. Learn more about what staff think about living and working at Lancaster[1], and supporting equality, diversity, inclusivity and family online.

Initial enquiries can be directed to Distinguished Professor David Hutchison (d.hutchison@lancs.ac.uk) or Professor Nick Race (n.race@lancs.ac.uk) for an informal discussion (scc-hr-enquiries@lancs.ac.uk); +44 (0)1524 510331.

[0]: https://hr-jobs.lancs.ac.uk/Vacancy.aspx?ref=A2588-R [1]: https://www.lancaster.ac.uk/new-staff/welcome/

We welcome applications from people in all diversity groups.

← PreviousPage 2 of 3Next →