HMH

Awesome, after reading through the first time contributions I can see my first albeit pretty minor patch is included in this release!

Overall contributing has been very pleasant and the people on Mozilla's matrix channels have been helpful getting me on track. But better than taking my word for it, you should try contributing for yourself :)

This would be massively annoying for my local network. I trust that I do not mitm myself for malicious reasons and really do not feel like setting up certs on my lan...

Talking about certs, nowadays it seems browsers want to make you believe that self signed certs are some diabolical work straight from hell. Using self signed certs with websockets on Firefox is actually nearly impossible [1].

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1187666

At the same time smart phones are also a great example for my point: Just consider soldered on batteries and the lack of software updates or in Apples case slowing down old phones.

I guess the reason why a lot of phones use gorilla glass is that the quality difference to regular glass is more noticeable and things like scratch resistance are a little more than just pure durability.

But I have to admit that my statement was of course somewhat hyperbole :)

> But when people say things like "capitalism has no need for X", what they are bemoaning is that the bulk of the market is choosing different tradeoffs.

What I am bemoaning is planned obsolescence, under capitalism there is hardly any incentive to make things more durable unless absolutely necessary. Superfest glasses may or may not be an example of planned obsolescence winning over durability as I do not know the costs to produce them and it seems there are some downsides like breaking rather violently once their time has finally come.

This reminds me of Superfest (German for super hard) glass [1] which was produced in East Germany. It was optimized for durability so bars, pubs, restaurants, etc. didn't need to constantly buy new drinking glasses. But sadly after reunification production ceased as capitalism has no need for drinking glasses that do not break.

[1]: https://digitalcosmonaut.com/2020/superfest-ceverit-glass-dd...

> attempting to do it that way is just re-accumulating technical debt left over from X11

I see, I do not know much about the internal workings of Wayland. The problem I do see though is that screen recording is an essential feature that has been neglected for years due to the lack of a standard.

> This is a really bad idea, don't do this.

While I agree that getting window positions may be questionable, providing a human readable title for the streams that have been selected seems obvious, yet is missing.

> syncing issues with window sizes being reported incorrectly

This seems rather theoretical to me, while of course true, I doubt it is going to matter in practice.

> It's also a misuse of the screencasting API.

Hmm, do you think there is a better solution, that isn't giving up and having each desktop environment do its own thing?

> If you want to do this right now then you'll have to integrate with the specifics of the window manager or shell. There is no way around it, I really doubt you will get much interest in doing it another way.

This is also my conclusion, alas it's not happening and one of the reasons why I think Wayland is not quite there yet.

> had to read the blog on lynx due to light grey text on white.

No need to go that far, this site supports reader mode.

> Does this change require GNOME and/or Wayland?

PipeWire is not bound to Wayland or a specific desktop environment, it runs (mostly) fine under GNOME, KDE Plasma as well as sway. The desktop interfacing is done via flatpaks xdg-desktop-portal, as long as your desktop implements that you should be fine.

> In fact in the first few releases of Fedora Workstation where we shipped PipeWire we solely enabled it as a tool to handle screen sharing for Wayland and Flatpaks.

PipeWire is truly a godsend for screen recording on Wayland, not very long ago, if at all, every desktop environment / compositor had their own home brewed protocol and developing anything related to screen recording on Wayland was a massive pain [1]. I still can not fathom how Wayland could even exist this long without any decent way to do screen recording and I also do not get why there is no official protocol or at least extension for this!

But thanks to PipeWire some first projects finally support screen capturing on Wayland. Both Firefox (via webrtc) [2] and OBS [3] directly interface with PipeWire. For my personal project Weylus [4] I opted for the gstreamer plugin as at the time the documentation for PipeWire itself was very lacking and I could not figure out how to get it to work. Fortunately gstreamer is better documented and it turned out to be rather easy to use. But it shows that all this is still in an early stage as I hit quite some bugs, probably the most egregious (maybe even somewhat funny) one is KDE's kwin_wayland crashing if you hover the cursor over any window close button while screen recording [5], sadly it doesn't seem to be fixed yet. For some more bugs I've hit, see [6].

Something I am a little worried about is that flatpak with their xdg-desktop-portal [7] is now responsible for the de facto standard how screen recording is negotiated on Wayland. I fear that this means if flatpak does not need a feature, even if it makes sense in another context, it probably won't ever be implemented. For example something I require for my project are proper window names and their position as well as size, but apparently this is nothing essential for flatpak and the issues I have opened have largely been ignored [8]. This is by no means meant as an accusation, just an observations that things are not all good.

[1]: https://github.com/H-M-H/Weylus/issues/3#issuecomment-652670...

[2]: https://webrtc.googlesource.com/src/+/refs/heads/main/module...

[3]: https://github.com/obsproject/obs-studio/blob/master/plugins...

[4]: https://github.com/H-M-H/Weylus

[5]: https://bugs.kde.org/show_bug.cgi?id=435042

[6]: https://github.com/H-M-H/Weylus/issues/3#issuecomment-808940...

[7]: https://github.com/flatpak/xdg-desktop-portal

[8]: https://github.com/flatpak/xdg-desktop-portal/issues/created...

OBS is truly a great piece of software. From what I can tell it is built in a pretty modular way, especially looking at plugins. Is anyone using OBS (or parts of it) as library for their project?

I have been following the development of screen capturing under Wayland a little and what I can tell you is that at least for Wayland OBS directly interfaces with pipewire (which is very much platform specific and low level), see: https://github.com/obsproject/obs-studio/pull/4287/files

Thanks for the insights!

> Best way to really check it is with high-speed video capture of a hardware display and the mirrored content in the headset, of a high-(temporal-)resolution stopwatch.

Agreed, it's a little cumbersome but afterwards you can be pretty confident in your result. Have you measured 3ms like that? Getting below 3ms with small deltas using NVENC sounds possible but considering that even with a refresh rate of 120fps there is over 8ms between two frames, 3ms does sound suspiciously low.

One last question, have you done any experiments with the video codec, due to compatibility requirements I am pretty much limited to H.264 but was wondering if there is something better for super low latency encoding?

> The trip from video encoding on the PC, over the WiFi, and display in the headset averages about 3ms for me, using a 5ghz access point that’s several years old.

Wow that's pretty impressive, especially considering the 4K main screen + additional virtual screens used. I wonder how you get the latency this low. 3ms means no buffering at all, neither during encoding nor decoding. The best I could achieve so far is a little less than 60ms but with the additional restriction that playback has to be done via browser.

This very much reminds me of "The Measure of a Man" [1], one of my favorite episodes of Star Trek: TNG. Just as in this actual case AI rights are discussed, albeit things are a little more dramatic than a ruling about patents. It has to be decided whether to Data, an android/machine, should be granted the same rights as to a person or if it is fine to dismantle him for research purposes without asking.

While I agree with the current ruling in the UK, this statement does not sit too well with me:

> "Only a person can have rights. A machine cannot," wrote Lady Justice Elisabeth Laing in her judgement.

In my opinion this sets a bad precedent in case we ever achieve artificial general intelligence (AGI) [2], which I think is perfectly possible, especially considering that we humans are nothing but complicated biological machines. And I think an AGI should very much be considered a person. That's why I think the way how a US judge in a prior cases put it is more agreeable:

> As technology evolves, there may come a time when artificial intelligence reaches a level of sophistication such that it might satisfy accepted meanings of inventorship.

> But that time has not yet arrived, and, if it does, it will be up to Congress to decide how, if at all, it wants to expand the scope of patent law.

But admittedly this is still all very hypothetical as I don't see AGI happening in the near future and for now there is no real problem.

[1]: https://en.wikipedia.org/wiki/The_Measure_of_a_Man_%28Star_T...

[2]: https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

Neat, the backend is indeed very simple. If you prefer to run things locally, just invoking `nginx -c /path/to/nginx.conf -e err.log -g "daemon off;"` [1] and your curl command seems to be everything required, save for httpbin.

[1]: https://gist.github.com/jvns/edf78e7775fea8888685a9a2956bc47...

I always wonder why something like xargs is not a shell built-in. It's such a common pattern, but I dread formulating the correct incantation every time.

I was happy to read that the author comes to the same conclusion and proposes an `each` builtin (albeit only for the Oil shell)! Like that there is no need to learn another mini language as pointed out.

You are right, I'd be pretty surprised if such a statement came from the government itself. But this complaint is actually only from some members of the parliament belonging to the Digital Agenda committee with the chairman being a member of the opposition.

Perhaps drawpile: https://drawpile.net/ is worth a look. I have seen people create rather large artworks with it, you even have support for pressure sensitivity. It does not run on tablets though as there is only a desktop version.

And that's why I switch to a sane language like python once my shell scripts get longer than a few lines.

But I would love a shell with a sane language. At the moment I am using zsh + oh-my-zsh because I can not let go of the autocompletion I get. I tried some other shells like oil shell, ion shell, nushell and elvish but sadly the completion is just not there yet. The only shell with maybe even better completions I came a cross is fish, but I don't love the language, while it seems better than sh/bash to me I'd much rather have something more similar to ion shell with stronger typing.

Thinking about command completion, this seems like the analog problem to editors and the language server protocol. Is there something like a command completion server?

Does anyone know what's going on?

Interestingly there is a term for the last known member of a species: Endling [1]. Scrolling through that Wikipedia article always gives me an eerie feeling. It looks like the Chinese paddlefish is not listed yet though.

[1] https://en.wikipedia.org/wiki/Endling

This is certainly fun to look at. Thinking about this, are there any programming languages that do i18n for their keywords?

I agree! Recently I have been wondering if there is any reason why for example Android devices with a camera don't implement the USB video class specification, except for the reason that nobody cared enough.

This is quite similar to my latest project Weylus [1]. You still have to create the virtual output yourself but then you do not only get a second monitor but all the touch events are handled properly and you can even use a stylus/pen with pressure sensitivity. Additionally it also works on iOS/iPadOS.

[1]: https://github.com/H-M-H/Weylus

Thanks for your feedback! I honestly did not expect Weylus to get that much attention. Unfortunately my time is limited and I am not too comfortable taking donations then.

> 1) From the video, latency would need to go way down.

On the encoding side the only thing I can think of to improve latency is to try hardware encoding if available. Something like NVENC might bring it down. That leaves optimization for the network protocol to use, something based on udp is likely better suited than websockets. And finally mobile apps probably leave more room for improving playback latency (and enable other protocols than using websockets).

> 2) Pressure support is important, if not critical.

It is pretty straightforward to add new input methods, I will add some instructions to the Readme, maybe someone will step forward and add support for macOS and Windows.

> 3) Packaging.

There already is a .deb, although I still need to fix some dependency issue [1].

> Next step would be to go from mirroring to second display.

Apparently this is almost possible right now (at least with X11), see [2].

> breadcrumbs

I think this is the way to go, I will add some more info on how to contribute soon.

[1]: https://github.com/H-M-H/Weylus/issues/6 [2]: https://github.com/H-M-H/Weylus/issues/5

I already used it the other way around with a hotspot created on my laptop. As long as you can connect your browser with decent speed it should work just fine.

Yes I have. I have decided to go with using a browser for now as I simply do not have the time to support multiple mobile apps and I am not too inclined to pay Apple 99$ for a Developer account. Personally I need this to work on my iPad, so you could consider using the browser as my way to break free of Apples walled garden :)

If someone wants to build a mobile app I am more than happy to provide other means of connection to Weylus than websockets. This also might bring the latency down as udp streaming and more control over video decoding and playing is possible then.

Yes I do! That is where I got the idea to use uinput from. But as this is no longer maintained and does not mirror the screen I decided to create Weylus. Something that is different in GfxTablet is that inputs are not directly mapped to screen coordinates but it works like a regular touch pad.

> ...and if vnc client can send a pen input to the host and host can see it as a proper HID device.

As far as I can tell this is not possible: https://tools.ietf.org/html/rfc6143#section-7.5.5

Also selecting a specific window, which is possible in Weylus, does not seem to work with vnc.

I have been using Weylus and Xournal++ a lot by now and video streaming is pretty stable (at least on my machine :P). In case the connection does fail a simple page reload will restart the stream.

> I'm guessing it's slow because capturing the display and encoding it takes a while?

This is a possibility but encoding is already pretty optimized (ffmpeg and libx264 with ultrafast as preset), which is faster than my previous approach of just streaming PNGs. And streaming PNGs was not that slow either, even on a rather old laptop. That's why I suspect there's something wrong on the tablet.

Regarding noVNC: What latencies do they achieve? I am not a webdev so I guess they might have some more tricks up their sleeves, I will probably have a look. As far as I can tell noVNC does not support things like multitouch and pressure sensitivity so that rules out just using it (at least for my usecase).