The 5.7 kernel is out | Hacker News Reader

The 5.7 kernel is out

lwn.net

210 pointsthuva4108 comments

> a new, Microsoft-blessed exFAT filesystem implementation

For those of us who lived through the "Linux is cancer" days, we are truly living in interesting times :-)

Extend phase.

Ehh, Microsoft only opened exFAT after smartphones dropped all SD card slots and it was no longer possible for them to extort Android device manufacturers with patent licensing.

How long until the patents expire?

~2009 + 20 years. You've got a long time to wait if you're not part of OIN.

They're arguably invalid patents, in the sense that the filesystems concepts within are not remotely novel and the application to FAT is trivial and obvious.

Released 2006, actually. Shipped with WinCE 6.0.

Doesn't matter when the code shipped, only when the patents were applied for. The application date on the later patents is 2009.

https://worldwide.espacenet.com/patent/search/family/0407899...

https://worldwide.espacenet.com/patent/search/family/0407898...

In a sense, it doesn't matter if the patents are valid or not, but isn't the rule that you have to apply within one year of making the invention available, and shouldn't including it in a release count for that?

(Maybe that rule changed with the big changes around first to file and expiration based on application instead of grant date.)

No, you are correct. Publicly shipping the product that implements the invention starts the clock. But here, the parents claim priority to an application filed in 2004. (That means the invention was disclosed in that earlier application, even if it wasn’t specifically claimed.) When you claim priority like that, your 20 years starts running from the earliest priority date. So the patents expire in 2024.

Around here there are plenty of Android phones and tablets with SD card slots to chose from.

I agree. It's a token gesture. Even if you do have SD card slots android offers to extends storage to that, which I think means it formats the card as EXT4 or something.

Maybe TV manufacturers care about this, but in mobile it's a non-event.

Maybe Microsoft could open-source MSDOS in order for FreeDOS to have a rejuvenating moment.

well, they have for early versions of msdos[1], though probably anything less than 3.3 only has historical value to anyone.

[1] https://github.com/microsoft/MS-DOS

Yes, they like to do these kind of things in the name of loving open source nowadays. Also open sourced a 40 year old BASIC implementation which can only be good for a Computer Museum display.

MSDOS 6.0 might be interesting. Windows 3.1 too.

MSDOS 6.0 might be interesting. Windows 3.1 too.

And MS-DOS 4.0/4.1, not to be confused with 4.00/4.01, which had multi-tasking features:

https://en.wikipedia.org/wiki/MS-DOS_4.0_(multitasking)

> after smartphones dropped all SD card slots

Last I checked, only at the high end and Google devices, and substantially less true in non-US markets.

Careful with proclaiming such a bubble to be universal. I am sure in mountain view some PMs make decisions based on "oh but we killed the SD card years ago" but they don't have actual power to do that.

You can also plug any USB stick into most androids

It's fun to see bubble-driven remarks like "no phones have SD card slots anymore" or "there are no headphone jacks on phones nowadays". Welp, you can influence that and I won't buy a phone without either.

>I won't buy a phone without either

You used to be able to say the same for phones without holes in their screen or TV sets without internet connectivity.

Or a phone with a removable battery. I am personally disappointed by that one. A lot of phones are skimping on internal batteries these days and it kills the life of the phone fairly quickly.

The previous phone I had, since I had been burned repeatedly by phones I was otherwise happy with but the internal battery didn't last a year, I decided to shop around for the few remaining Android phones with decent performance that had an easily replaceable battery. I had to pick a model from late 2016.

It's more the ubiquitous and uncontrollable fast-charging that kills batteries. I would love for charging power controls baked into Android, as I know there are one or two manufacturers who have them (ASUS and Sony?).

Personally, I don't have a problem if the battery isn't easily removable. I don't think I'll find myself in the situation of needing to hot-swap batteries on the move before I get home, today's battery technology has ensured sufficient energy density to make all-day long battery life a reality.

What I ask of manufacturers is that at least the battery isn't glued or soldered in, and that there are clear instructions on how to unscrew the back of my phone to replace the battery after it starts dying out. That, and make it easy to take off the back (no adhesives (looks at you apple), and no taking out the front screen to access the battery), and make it easy to buy a genuine battery from the manufacturer's website.

I think that's a good compromise of thinness, ease of manufacture and ease of replacability.

> today's battery technology has ensured sufficient energy density to make all-day long battery life a reality.

That is a sad comment.

Yesterday's battery technology ensured sufficient energy density to make _all-week_ long battery life a reality. Yes, those were much weaker phones computationally, but I'm fine with staying with just phone calls, SMS and calendar when I'm low on battery, for an extra week...

In some aspects, smartphones have become toys. Scrolling through reddit/facebook/twitter doesn't really make it worth having a smartphone, since those are all dumb activities.

However there are certain features that a smartphone gives which I wouldn't give up. Navigation with google maps, being able to do banking services on the go with my bank's app, being able to review flashcards with the Anki app, having my railcard/tesco clubcard/nectar card on my phone and never losing it, being able to check any important emails, checking the news if something breaking happens and I need to know instantly.

Heck, we praise Apple for bringing services like calling an ambulance for you if your pulse shows a certain pattern matching a heart attack (via the apple watch), and we also praise smartphones for actions like alerting people for emergencies like a terrorist attack or issuing weather warnings. I wouldn't give up any of those features just to have a dumb nokia phone with a week-long battery life.

Highly recommend you check out this: https://www.blloc.com/

Why give them up? Have 75% of the battery life be used for all of those activities, but build the phone so you could still make calls and send text messages for a longer while later.

Also - Smartwatches are silly IMHO; and people can be alerted just fine with a voice or text message.

Camera/camcorder makers are huge market, and they use Linux internally. E.g. Sony's stuff: https://oss.sony.net/Products/Linux/

What a fastidiously/meticulously organized wall of text. I want the picture book version :(

Patents should not be allowed on communication protocols and storage formats, or anything with lock-in potential really.

Hear hear!

> after smartphones dropped all SD card slots

So companies like Samsung[1], Sony[2], Motorola[3], Xiaomi[4] and LG[5] don't make smartphones anymore?

[1]: https://www.gsmarena.com/samsung_galaxy_s20-10081.php

[2]: https://www.gsmarena.com/sony_xperia_1_ii-10096.php

[3]: https://www.gsmarena.com/motorola_edge-10133.php

[4]: https://www.gsmarena.com/xiaomi_mi_a3-9769.php

[5]: https://www.gsmarena.com/lg_k61-10088.php

Finally, we get a non-journaled semi-cross-platform filesystem. fat32 just wasn't cutting it... /s

When will we get a proper performant journaled cross platform filesystem that doesn't require additional software on Windows, MacOS, and Linux? I can't believe we still don't have anything like that.

Btrfs and WinBtrfs. The file system is the journal. https://github.com/maharmstone/btrfs

Previous discussion on WinBtrfs https://news.ycombinator.com/item?id=15177002

Journal is about obviating fsck, just do journal replay to make the fs consistent again. I think for most media like USB sticks, the most platform interoperable file system is UDF. The gotcha that trips most people up I think, is that you shouldn't partition the media device. Just format the whole block device as UDF. I do this with mkuddf using defaults and it works fine across all three operating systems with similar performance as FAT32.

I think the complexity of btrfs and its mix of device and file system layers rules it out for general purposes.

Why?

I use it on my laptop with 0 issues. It’s really not that complicated...

It's not about that - the filesystem needs to be simple snough that multiple vendors' implementations will be compatible.

Remember SOAP interoperability? Me neither. It was too complex for its own good. That's why REST is used everywhere today and experienced web developers smile every now and then when they remember they don't need to use it anymore. Well, most of them at least...

I’m not talking about complexity to use but rather architectural complexity. Rather than being a simple fs driver, it (in the ZFS model) consumes raw devices then re-exposes them as software-defined volumes optionally with a file system api already exposed or as software defined block devices that can then be formatted with another filesystem.

It basically upends the entire separation of layers and requires a completely different approach to use.

Btrfs actually has a simpler interface than ZFS: it consumes block devices and exposes only a filesystem, not more block devices. It does have integrated volume management functionality, but overall acts much more like a "regular" filesystem than ZFS.

The file format is very simple. The write guarantees of the file system for full OS usage may not be simple, certainly they're not simple on Linux, and Btrfs development shows that this is hard.

The GRUB Btrfs driver is pretty small considering the Btrfs feature set. And yet it understands all the multiple device stuff, the logical and physical device abstraction, and snapshots and subvolume navigation.

I like always on checksums for every block, both metadata and data. It's a good fit for the cheaper flash media out there including USB sticks which non-deterministically report back garbage, rather than a discrete read error.

With WSL, shouldn't it be possible to use any native Linux file system?

Only sort of. Windows communicates with WSL via 9p, which has decent performance but is still a network protocol. It certainly isn't possible to mount an ext4 filesystem on D: with native performance.

Does this mean I can use libvirt to expose a directory on my system via 9p and use it within Windows?

Maaaybe. Hadn't considered the option, honestly.

>Finally, we get a non-journaled semi-cross-platform filesystem. fat32 just wasn't cutting it... /s

Hope you never had to store files larger than 4GB otherwise FAT32 would not work for you. (Before you respond with "that's what archive files / files splitting are for", not every device that takes files on an SD card and supports FAT32 has a way to join files on the fly/a way to read archived files.)

Does UDF count?

My favourite part of this (source: https://kernelnewbies.org/Linux_5.7#New_exFAT_file_system) is that they originally went with an MS-produced exFAT driver, but dumped it because even MS said the Samsung one was better.

Given that Samsung has had years to perfect theirs, and it was an ms licensed implementation, that’s not really surprising tbqh.

(To clarify, this is in reference to the Linux driver implementations from both and not in comparison to the Windows version.)

That's not accurate. The initial implementation (in Linux 5.4) was also a derivative of Samsung exFAT code, however, it was not contributed by Samsung directly and it was based on a much older version. Samsung then suggested¹ it would be better to upstream their latest exFAT driver (sdfat), which is what has happened now - after improving the code to be suitable for mainline.

¹ https://lore.kernel.org/linux-fsdevel/003701d56d04$470def50$...

Seriously curious: Is there better looking mailing list viewing software thats "free"? I loathe stumbling on one of these links.

Not sure if this answers your question, but if it really bothers you, you could change the CSS rules that your browser applies to a specific site. If you are using firefox, the file to edit is `chrome/userContent.css`.

I know it doesn't help others, but in my experience people using such software actually like this retro look (and I agree it does have some appeal to it - function before form).

I think my issue is more with navigation than style. Like Seeing the whole thread tree inline would be nicer no, even for the folks who still want the plain text feel?

There was Gmane. Too bad nothing had replaced it. It was the de facto site used when sharing links to mail list posts and it did have thread navigation that made sense.

Not to rain on the parade but it's exFAT, not NTFS.

How is this for shared storage among dual boot? Terrible idea?

It's a business investment, nothing more. There was a recent story about how M$ treats open-source devs (AppGet vs WinGet). If they could EEE Linux, they'd do it.

Not mentioned in the change summaries as far as I can tell, but this contains a (for me critical) fix to the intel graphics drivers (i915): https://gitlab.freedesktop.org/drm/intel/-/issues/1600.

Will this fix my "i915 0000:00:02.0: Resetting rcs0 for hang on rcs0" that's makes my screen freeze for 5 seconds multiple times a day??? It's been driving me slightly crazy.

Depends on the model. Quite a few were hit in 5.4, including mine, but was fixed in 5.5. I haven't seen a problem with 5.6 or 5.7 so far.

There is a fix for this available. I believe it's in 5.5. My distro (mint) doesn't have it in their kernels utility, so I'm sticking to 5.0.0 until they do.

(Note: It took me ages to find any information about this - https://gitlab.freedesktop.org/drm/intel/-/issues/673 is where the bug is supposedly fixed)

We can't know the answer. What you are describing is a GPU hang and GPU hangs are a whole class of bugs, it pretty much means "the hardware was misused and now it doesn't know how to move forward, so we reset it" (and the reset is why you get the 5s freeze, your graphics card was essentially rebooted). When this happens, i915.ko creates a nice file in your file system that you can use to create a bug report. Please check dmesg and do so.

Thanks for the overview. I know HN isn't the place for this but just for the record: there are no other i915 messages in dmesg (certainly nothing about a file with debug info).

This is on Ubuntu 19.10, kernel 5.3.0, and it started happening after a minor update, so I figured it would probably get fixed in another minor update. That didn't happen, so then I thought I'd better update to a newer kernel before reporting anything. Then 20.04 was almost here, so I decided to wait for that. Hopefully I'll get around to that soon and discover my particular bug has been fixed, otherwise I'll try to report it.

Seeing the hang message without anything else is surprising to me. I don't know why you're seeing that.

You can try kernel-ppa [0] to see if it's fixed in newer kernels, it's very simple and you don't need to compile anything. I would recommend you to test 5.7 from there, and if that doesn't work you can try drm-intel-nightly or drm-tip: those are the upstream graphics trees. If you report a bug, the first thing the devs are going to ask you is test these trees, so it might be worth trying before you even open a bug report.

[0]: https://wiki.ubuntu.com/Kernel/MainlineBuilds

Just a quick followup: I installed 5.7 (still on Ubuntu 19.10) and no hangs yet.

I got a bunch of warnings about missing firmware so I manually downloaded some i915 firmware files from the linux-firmware repo. I assume that's just an Ubuntu packaging quirk. I'm not sure if they were actually needed or not.

I had the same problem on Ubuntu 18.04, terrible lag spikes and similar messages regarding i915. After installing the least new kernel (5.3.1), it solved the issue: not one hang since.

Not yet finished kernelnewbies summary of changes: https://kernelnewbies.org/Linux_5.7

Oh, already? I got a patch in to fix PTRACE_SYSEMU on aarch64, but I still have a few patches queued up to fix a few more issues there. I had hoped to get them all into the same release so there is only broken and not broken releases, rather than broken, semi-broken and non-broken releases. Oh well, ptrace on aarch64 has a few other big issues anyway that wouldn't have been mergeable during the RC window, so I suppose I should just try to get that in for 5.8.

I hate to break it to you, but there are only semi-broken releases ;)

Well aware, but I was hoping to say to people "Just use 5.7 and it'll be fine". At the moment I have to say "Don't use anything prior to 5.7, unless you have a recent stable backpoint. In either case, some things are fine others aren't - I'll hope to have it fixed by 5.8". Which is fine, but more complicated ;).

Just CC stable@ and get them into the next point release?

Was curious; quoting https://www.man7.org/linux/man-pages/man2/ptrace.2.html:

> PTRACE_SYSEMU, PTRACE_SYSEMU_SINGLESTEP (since Linux 2.6.14)

> For PTRACE_SYSEMU, continue and stop on entry to the next system call, which will not be executed. See the documentation on syscall-stops below. For PTRACE_SYSEMU_SINGLESTEP, do the same but also singlestep if not a system call. This call is used by programs like User Mode Linux that want to emulate all the tracee's system calls. The data argument is treated as for PTRACE_CONT. The addr argument is ignored. These requests are currently supported only on x86.

That sounds like a fun pile of dragons. Good luck backspacing the last bit out soon ;)

OK so let's say that I want to run a relatively stable linux install (so not the arch bleeding edge methodology), but I do want to run the latest linux kernel, or as close to it as possible.

What are my choices? Or is the only choice to alter my interpretation of the word "stable"? :-)

See if your preferred distribution has a kernel backport available. Most have backports of the kernel because it has relatively few dependencies and is backwards compatible, making it easy to backport.

E.g. Debian currently has a 5.5 kernel in buster-backports[1]. However, Debian's backported kernel doesn't get the same level of attention and isn't subject to the same security policies as stable releases, so may be lacking some security patches that the stable kernel has.

[1]: https://packages.debian.org/buster-backports/linux-image-amd...

I find NixOS is good for this - but it depends partly on what userspace programs you need.

You can run NixOS Stable, which is on a six-monthly release and testing schedule, and add the latest kernel, very easily.

+1. NixOS benefits especially, because a broken kernel/nvidia update never renders your system unusable. You can make changes to all your heart's desire, and if something does break, all you have to do is reboot and select previous configuration

I don't know enough about Linux to offer a complete answer, but I can offer the distribution I use as a suggestion. I've been using Solus (getsol.us/) for a few years and I think it comes close to what you're asking for. The kernel version is currently 5.6.13.

There's no i3 flavor installation image, but you can grab it from the software center/eopkg.

> Or is the only choice to alter my interpretation of the word "stable"?

Depends, if your "stable" means has been tested by tens of thousands of users under different scenarios, then running 5.7 right now is by definition not stable.

But if stable means that it is unlikely to cause problems, then even Arch fits that definition, which honestly is also suitable for Windows 10 and macOS at this point.

Anecdotally, I have an Arch install going strong on 5 years now, so even Arch isn't unstable, but it does require paying attention to the announcements.

I'd say a good compromise may be to wait a couple 5.7 point releases in. By then, practically any distro should have it.

Manjaro is a little behind Arch to offer more "stability", there's the OpenSUSE rolling-release option, Solus, Sabayon...

Fedora is the distro you want. It's not bleeding edge, but everything is up-to-date and the kernel is 1-2 releases old. It has a 6 month release cycle and 13 month support window iirc.

I find it a lot more stable than arch and Debian Sid.

I use Fedora with dnf automatic[1] on all of my machines. Only problem is my Desktop PC, which has a nvidia gpu with the official nvidia driver installed. Every few weeks there is a new kernel version which often only works with the brand new nvidia driver version. I use grub to choose the previous working kernel version until I take the time to download and install the new nvidia driver.

[1]https://dnf.readthedocs.io/en/latest/automatic.html

Ubuntu should have a PPA for that

Yup:

https://wiki.ubuntu.com/Kernel/MainlineBuilds

https://kernel.ubuntu.com/~kernel-ppa/mainline/?C=M;O=D

Technically, it's not a "PPA" and isn't usable as a source, but you can download builds and install them.

https://github.com/pimlie/ubuntu-mainline-kernel.sh is a decent script that wraps managing updates and installs.

Use the distribution of your choice, but with the most recent mainline kernel. The releases are already gated by the LKML process, so don't worry about actual, non-pre kernel releases being too unstable for any normal desktop/workstation usage.

If you don't lack the skills, and are ok with the system telling you when you need to spend some time on transitioning configurations/library versions, consider using Arch Linux. It will demand manual configuration to some extend, but that's mostly just enabling (and rebooting/manually-starting) systemd units for things like a GUI login manager. The benefit is, that you don't rely on backports for bug fixes, allowing things like youtube-dl (interacting with uncooperative websites by ~scraping) to work from the official repositories.

> Or is the only choice to alter my interpretation of the word "stable"? :-)

I'm not experienced enough to give you an answer, but a hearty "bravo" for recognizing this as an option. Too many do not.

For my own systems, I use CentOS + the ElRepo [1] ML kernel which is about a week behind upstream and uses the same compile options as Redhat uses. I would consider it stable for hobby use. I would not use the ML kernel in production systems unless you had a very specific niche need for it. It's assume you are updating the kernel weekly with ML.

[1] - https://www.elrepo.org/tiki/tiki-index.php

Ubuntu HWE?

Debian testing is on 5.6 right now. If you can handle a high update volume and are comfortable managing occasional packaging issues, it’s a rather nice “not arch but newish” experience.

Agreed. I was following Debian mainline releases but grew tired of out of date packages and putting my hopes and dreams into *-backports. After migrating to testing, it's been smooth sailing for well over a year and I've grown to appreciate a rolling release cycle.

Why not Arch? It’s been super stable for me over the past months I’ve been using it, and I configured to have Btrfs take snapshops on every pacman update, for peace of mind.

Haven’t had to roll back once.

On Debian/Ubuntu, you can just download and configure the kernel as normal, and then run "make deb-pkg" to get a .deb that plays nicely with dpkg. You can run whatever kernel you want without having to upgrade any user-space packages or depend on third-party repositories.

So I run Linux mint on my personal laptop, but boot a mainline kernel built from source (so more stable userspace, but cutting edge kernel, as I do a fair amount of kernel development). I think that sounds like they setup your asking about?

Building and installing your own kernel isn't too bad. If you want to give it a shot:

1. download mainline, ie torvalds/linux 2. make -j localmodconfig olddefconfig 3. make -j 4. make -j modules_install install

and reboot.

If you apply the appropriate Debian[1] or Ubuntu kernel patch sets, you can build .deb packages of your kernel and ensure that the appropriate kernel settings and integrations are set for your system.

[1] https://salsa.debian.org/kernel-team/linux/tree/master/debia...

Upstream kernel has make deb-pkg target for creating deb packages. no need for patching.

Ubuntu + the latest Ubuntu kernel patch sets. I have a script that downloads the latest kernel source, applies the appropriate patches and builds it if you're interested.

I would recommend the latest Fedora release. The kernel is updated frequently, seem to be only a couple weeks behind upstream. Fedora has been incredibly stable for me.

Piggybacking on this, do any of the modern distributions treat kexec as a first-class thing? I would prefer to never reboot, but I also don't want to keep running ancient kernels.

I update CentOS boxes kernel using "ElRepo" and works out fine, for my use case (webservers, etc)

Linux* kernel

*GNU/Linux kernel

No, the kernel is just linux

Kernel of the GNU/Linux system. /s

> was this one deprecating the long-standing 80-column limit for kernel source

Amen!

Let's try to get rid of the "80 column limit" dogma and move on to more flexible and realistic limits, in all languages.

Apparently it's been moved up to 100, and 80 is still preferred [1].

I have a personal bias toward 80 because I like having multiple buffers side by side (100 would also work).

I always assumed that the hard limit on column length was as much excuse to reject sloppy merge requests as anything else: I don't mind a few 150 column lines, but if every single line has to wrap a 100 column buffer there's something wrong.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

Bpf-lsm is a welcome addition. I can see a new generation of endpoint defense tools built on it.

[Meta] what's with all the auto-collapsed comments.

And once again, no NVIDIA drivers but lots of AMDGPU stuff. How are people still buying NVIDIA?

Machine Learning.

A much farther developed ecosystem for most/all aspects of computation, including lots of free software. I know, nVIDIA are terrible with their close-sourcing of almost anything and their OpenCL betrayal. But AMD is too much of an inferior option for doing compute work.

> +However, never break user-visible strings such as printk messages because +that breaks the ability to grep for them.

Amazing it took this long for such a common sense formatting method to be acceptable.

This is not a new rule, it's just a rule that was not written there. You can probably find emails from Linus with nsfw-language bashing people for breaking messages.

The rule was already written there. The patch just reformatted things a bit.

Reply on news.ycombinator.com