FreeBSD 13.0 Beta1 Now Available
lists.freebsd.org
lists.freebsd.org
I, for one, learned quite a few things too out of your presentation, which is here: https://2019.eurobsdcon.org/slides/NUMA%20Optimizations%20in...
And we have John Baldwin to thank for upstreaming the Netflix kTLS.
It seems like encrypting the video stream traffic at this kind of scale requires a substantial engineering effort and special hardware.
Encryption also means that shared downstream caches at ISPs can't do anything to reduce their bandwidth consumption. The only option is to contact Netflix and physically install one of their edge caching boxes.
Content tampering protection could be obtained by simply computing content hashes offline once, which the clients could verify.
True there's no out-and-out porn, but there is content that's close to the line, and beyond that there are themes that could be indicators that a viewer is gay, or in some other demographic that he might want to keep private.
Target can tell if you're pregnant based on your shopping; certainly there's a lot that can be inferred from the movies you watch.
I am very skeptical about that story, it sounds too good/scary to be true, and there is only 1 reported incident, without proper sources. Sounds made up to me.
https://medium.com/@colin.fraser/target-didnt-figure-out-a-t...
https://www.xda-developers.com/android-permissions-bypass-pl...
https://www.trustedreviews.com/news/facebook-scraping-call-m...
They are completely terrible on privacy, lol, the answer here is "if you care about privacy don't use a samsung". Or more generally "don't use android".
If you care about privacy, you don't buy Samsung phones (or other products like TVs). They are the tip of the spear on data collection.
Also while HN likes to raise the spectre of TVs connecting to open wifi/shipping with 5G radios, at the moment there is no evidence for either so users could always use a trusted device to play back Netflix rather than the TV app and leave the TV without internet
Like let's say you're a network admin of a college with conservative religious views, and you want to see if anyone in the dorms is watching "immoral" content. You probably can just intercept an entire unencrypted session and replay it on your machine and see what it was. But you don't really have the funding or access to expertise to develop a side channel attack yourself, and there are no off-the-shelf devices that will do this for you, are there?
Encryption is likely the difference between your management saying "Show me what the kids are watching" and "This isn't worth assigning our network admin to spend half a year on effectively a cryptography research problem."
(Incidentally, encryption may also be what allows a sympathetic network admin to refuse an order from their management, which is also worth considering in your threat model.)
I think it's true that if you had either the resources of one of the richest handful of countries in the world or access to some talented grad students etc., you could do it. But if you're even a non-rich country (like one of the many small countries with moralistic governments that censor the internet) it seems harder, and if the goal is spying on what people watch, it's unlikely that people talented enough to do it will find this a problem they're happy to volunteer their time to solve.
(This is a genuine question - the attack might be much easier than I think!)
TLS fixes a whole lot more than just privacy, it's also authenticating the remote end. Are we really suggesting dumping something that is trivially accelerated in hardware to do some homebrew crypto crap just for the sake of a forum thread?
Netflix solution is fine, and the concentration of interest in TLS means it only gets cheaper over time to build Netflix-like configurations, which is especially great since we've spent the past decade or more trying to convince the entire industry this configuration is also best practice
But yeah, you’re right. You could glean a lot of information from nothing but a collection of movies’ exact runtimes, as visible from the network stream. Although that wouldn’t tell you much about a single movie, given enough viewings you could make pretty good guesses about which movies someone is watching.
Such as an end user device asking for 4k, but a middlebox somewhere between the customer and Netflix inspecting the http content and blocking it (or modifying the client request in flight), because "only 480p allowed here" or something like that.
But, beyond that, the battle to use http when it makes sense has been lost. Modern platforms strongly encourage https and discourage http.
I'm genuinely curious: why did you implement it in FreeBSD rather than Linux? Isn't that kind of offload something that already exists in Linux? Or does something in the way Linux is implemented/designed that makes it impractical / impossible to implement as efficiently? Or is somehow Linux not secure enough for your use case? Or not something else enough?
I'm asking because I'm always interested to know what are the practical limitations of Linux and also because, while I think it's nice that Linux had some serious competitors, I cannot help being a bit sad that contribution like this will only benefit the few people that use FreeBSD (even if to be fair, it seems that it will benefit all netflix customers at least, which is quite a few people ;) )
Same could be said the other way around, the thing is Linux can implement it and even take the code, the other way is rather difficult....so much for freedom.
And not sure what you mean with "few" people, the few Playstation 3/4/5 users? Juniper? OpnSense/Pfsense installations? Or the few Truenas/Freenas users?
>Or not something else enough?
The point of the licence is a good point. Is the kind of code drewg123 is referring to regularly copied from BSD to Linux? (i understand it should be possible in this direction?).
Thanks for the video, looks like exactly what I was looking for, I will watch it later.
However, publicizing the ideas and the results is valuable to other operating systems even without the code. If you're building something on Linux (or Windows? or ?) that could benefit from kTLS (including NIC accelerated kTLS), knowing it can work and having a roadmap is great. You would still need to do the plumbing, but you could skip a lot of the design. Being able to look at the code is nice too.
And, if you're willing to try FreeBSD + nginx, you could jump directly to that. If you're deploying something with similar performance characteristics to a CDN box, it's probably a very narrow application, and doesn't need to run on the same stack as the rest of your fleet.
Not to mention all the Minix and NucleusRTOS Users ;)
Edit: It is! neat!
# uname -a
FreeBSD freebsd 13.0-BETA1 FreeBSD 13.0-BETA1 #0 releng/13.0-n244471-638e531019f: Fri Feb 5 17:12:07 UTC 2021 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64
# zfs version
zfs-2.0.0-FreeBSD_gf11b09dec
zfs-kmod-2.0.0-FreeBSD_gf11b09decEdit: Compared to the version of ZFS that was in base FreeBSD 12.x I mean
The installation was much easier compared to Arch, almost all packages that I wanted I just got with `pkg`. With using sway and wayland, the experience is really really fast and snappy even with this old laptop. At least compared to OpenBSD and xorg, the desktop experience here just flies.
It has been a good Sunday so far...
This will kinda help you:
Description from the page itself:
up.bsd.lv is a proof-of-concept of binary updates using freebsd-update(8) for FreeBSD 13.0-CURRENT and 12-STABLE to facilitate the exhaustive testing of FreeBSD and the bhyve hypervisor and OpenZFS 2.0, and to help elevate ARM64 to Tier 1 status. Updates are based on the SVN revisions of official FreeBSD Release Engineering weekly snapshots.
Features:
- Kernels are built with the GENERIC-NODEBUG configuration file
- /etc/freebsd-update.conf is modified to use up.bsd.lv
- /usr/sbin/freebsd-update is modified to "xargs" parallel phttpget (Thank you Allan Jude)
- freebsd-update(8) does not output to the $PAGER
Hope that helps.
But OPNsense is based on HBSD 11.2 ATM
>The latest release is based upon HardenedBSD 11.2
``` FreeBSD router.local 12.1-RELEASE-p12-HBSD FreeBSD 12.1-RELEASE-p12-HBSD ```
[1]: https://svnweb.freebsd.org/base?view=revision&revision=36816...
# uname -a
FreeBSD freebsd 13.0-BETA1 FreeBSD 13.0-BETA1 #0 releng/13.0-n244471-638e531019f: Fri Feb 5 17:12:07 UTC 2021 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64
# kldload if_wg
# kldstat
Id Refs Address Size Name
1 30 0xffff000000000000 11fd228 kernel
2 1 0xffff0000e1800000 24000 uhid.ko
3 1 0xffff0000e1824000 24000 usbhid.ko
4 1 0xffff0000e1848000 25000 hidbus.ko
5 1 0xffff0000e186d000 25000 wmt.ko
6 1 0xffff0000e1c00000 452000 zfs.ko
7 1 0xffff0000e2052000 52000 if_wg.koThere seemed to be some talk of Linux I noticed in past years, and I’ve been thinking about trying to out again, but I’d feel better if I knew that it was still well-used.
For info, a key thing about Tier 1 architectures is that "Binary updates and source patches for Security Advisories and Errata Notices will be provided for supported releases" (https://docs.freebsd.org/en_US.ISO8859-1/articles/committers...), which would make maintaining ARM-based SBCs much easier (no need for setting up cross-compilation toolchains or for very slow compilation on the SBCs) and could make FreeBSD an even better OS for IoT stuff.
https://lists.freebsd.org/pipermail/freebsd-arch/2019-Decemb...
I made some significant speedups a couple years ago, but I'm sure there's still room for further improvements, especially if you have hardware which I didn't test with.
(In fact systemd and launchd are so tailored to their respective operating systems that FreeBSD likely would have to do their own init replacement if they cared to.)
That is not a good talk unless you already advocate for abandoning standards, and celebrate Linux's growing power to dictate how the shrinking number of remaining OSes design things. You might easily miss it if you're already onboard with systemd, but in his point by point address of systemd's shortcomings - he repeatedly handwaves and redirects.
Systemd is a mess, which is an amazing accomplishment considering the fact that it is supposed to be this great unifying/simplifying layer. So awful that the DoD held out for the longest time in formalizing any kind of baseline security audit procedures that address it. Compare that to something like Solaris SMF. I've never had a problem with the rc system that made me blame the underlying design, but if I did - systemd would be at the bottom of a very long list of my potential solutions, right under 'Set the machine on fire and pickup a copy of "Industrial Society and Its Future"'.
I really like the guy running the OpenZFS project, but it is painfully obvious that it is only going to increasingly cater to Linux - and coincidentally a lot of the really annoying aspects of the project are a result of that. For example: that Rube Goldberg build system, with massive amounts of code duplication... that wouldn't be there if not for autotools and gnu export symbol games.
And by the way standards aren't really the word of some kind of god that you should follow blindly and never question.
Many standards are ratified just to settle the status quo regarding various solutions to a common problem.
At this point, one might even argue that the next "standard" for init systems should just be based off systemd.
Only some Linux distros use SysV init and they're not any of the big players (Fedora/RH was the distro creating systemd), Ubuntu adopted systemd as the default 5 years ago, SUSE did the same 6 years ago, Debian 5 years ago. Almost everything else is a hobbyist OS (I've never seen Gentoo or Slackware in production anywhere else than at small companies where the server was on the critical path for maybe several tens of thousands of dollars, at best, which is peanuts on a global scale) or a niche OS (Alpine is used 95% as a container distro, and who freaking cares about the init system in a container? :-) ). Basically anyone who "puts there money where their mouth is" uses systemd.
SysV init was never adopted by BSD.
I think Solaris used to use it but it's been using Service Management Facility for the past 16 years.
HP/UX and AIX use SysV init, I think, but they're so vanishgly rare that 99% of developers and sysadmins out there could have the most successful careers in history and both HP/UX and AIX could be nuked from history for all they care.
And my point was that nobody even bothered to standardize SysV officially, though it was supposed to be a standard. That's how little everyone cared.
Poettering might be annoying, but he cares, he's opinionated and is actually knowledgeable. I've been using his software for a while and it's quite solid and his vision makes sense. POSIX isn't enough for a modern operating system.
> POSIX isn't enough for a modern operating system.
So there should be no standard? I can think of only one time I ran into a shortcoming that can be blamed on POSIX, and a few more where the OS just did a bad job in implementing the standard. Can you imagine how awful things would be right now if the DoD hadn't forced the concepts of interface commonality and multi-source procurement? You'd likely be reading this on a Honeywell glass TTY hooked up to an IBM timeshare. Because without standards the network effect acts as an insurmountable barrier to entry, and incumbents fully control the height of that barrier with needless complexity and arbitrary breaking changes. Hell, not that long ago there was talk about wielding GPL export symbols as a weapon to punish nvidia... I'm no fan of nvidia, but I like the idea of an API inspired by spite a lot less. Also, in a world where your vision prevailed, there would be no AMD - because Intel already set the industry "standard", so why worry about second-source availability?
Which one? I just listed the others and they all use different init systems...
> So there should be no standard
No, they should turn systemd into a real standard. ECMAinitd :-p
> No, they should turn systemd into a real standard. ECMAinitd :-p
Well systemd can only work on linux, by design. So every standard compliant init system would require that it package a linux kernel. That is not only moronic, but with history as our guide: would directly result in never patched, network active, embedded software.
Yes, I’m for abandoning pointless standards. There are concrete benefits to systemd - faster boot, comprehensive service management, power savings, memory savings, a consistent interface to changing system state. What is the benefit to sticking with an rc script architecture designed to run a handful of processes on a pdp?
You've never written portable software, have you? You can't write portable software without a common interface. Have you ever looked at what autotools vomits out? You'll see shell scripts containing the likes of "echo $1 | sed 's/^x//'" in order to just get things to where they have a chance of sharing enough commonality to compile. Imagine how much worse it would be without POSIX. Nothing would run on anything that the developers didn't account for, this includes OS (plus version), environment, and hardware architecture. That is the world you are asking for, not the one we presently live in.
> What is the benefit to sticking with an rc script architecture designed to run a handful of processes on a pdp?
Freedom, for not only the end user but all the way up to distro packager. Systemd is designed to be non-portable, and force a network-effect pressure. I was similarly suspicious about WSL designing toward a linux API instead of POSIX... an effort to reduce the potential for alternatives emerging in the future, where your choices are windows or linux.
It was also not unique in parallelising startup tasks. Even sysvinit can be configured to do that with insserv+startpar.
Besides boot time on any init system with a fast ssd seems to be pretty damn quick. Saving 10 seconds every month will take a long time to pay off.
It's a toll that I decided to accept, but most people aren't aware of that.
For example this now admittedly dated review seems so suggest a drain of about 0.6% per hour.
https://lifehacker.com/how-much-battery-life-does-sleep-mode...
However the internet is seemingly full of complaints that machines drain fast during sleep so your experience may not be unusual but it might pay to check if something is wrong or at least sub-optimal.
For example GPU not actually turning off, usb device keeping it from sleeping, hardware or software waking up your machine.
Otherwise, I'm not aware of any easy wins (unless you're running 10.1 or older, in which case set hw.memtest.tests="0" to disable the boot time memory test which is very time consuming if you have a lot of ram and doesn't seem to be very useful; it was disabled in 10.2 and later).
If you can see what steps are particularly slow, some of them might be influenced to be a bit faster (eg, if it's waiting on usb scan to mount disks, you can make that time out sooner if it's not necessary on your system; you may be able to do background dhcp instead of synchronous dhcp, etc). But I think someone could spend some real time making it faster to the benefit of those who boot often.
Edit: following in the footsteps of cperciva's work; especially the profiling setup.
Native client: no. The web client is useable though, and virtualization can work: https://www.davidschlachter.com/misc/freebsd-videoconferenci...
As others have mentioned the comparatively slow boot is because (for simplicity's sake) the process isn't parallelized. You can install alternate init systems from ports if you want to speed this up.
I used Zoom on FreeBSD. By use I mean I was able to be in a meeting, see screen share, share my screen, but I had to dial in for audio and never tried webcam.