GStreamer 1.20: Embedded and WebRTC lead the way | Hacker News Reader

GStreamer 1.20: Embedded and WebRTC lead the way

collabora.com

176 pointsmfilion31 comments

This is good news, although it looks like GStreamer's WebRTC plugin is still in its "bad" plugins repo:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/t...

From the README:

> gst-plugins-bad: a set of plug-ins that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. If the blanks are filled in they might be upgraded to become part of either gst-plugins-good or gst-plugins-ugly, depending on the other factors.

This was a bit of a turn off when I was evaluating GStreamer for a WebRTC based product I'm working on.

The gst-plugins-bad repo is defunct. GStreamer now uses a monorepo (also mentioned in the article).

https://gstreamer.freedesktop.org/documentation/frequently-a...

That said, webrtc is still in -bad: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/tree/ma...

Thanks for the link to the new repo, I had missed that.

What did you end up using for WebRTC? Any specific things that you needed when looking at WebRTC implementations?

Hi, Sean! Thanks again for creating Pion. That's what I'm using for the product now. I shared my reasons for choosing Pion on Reddit when you posted about the Pion v3 release:

https://www.reddit.com/r/golang/comments/kj1net/pion_webrtc_...

One year ago… time seems to be speeding up for me.

Thanks for the kind words. Hopefully you are happy with your choice! If you ever need any help or just a WebRTC sounding board I am always excited to chat.

I'm not the commenter you are responding to, but we've been using webrtcbin in a production system since late 2019. We had some help from Collabora's consulting services (specifically from Olivier and Aaron!) and our use case was pretty targeted (near real time broadcasting of a video stream to a handful of internal users) so YMMV.

EDIT: Didn't realize you were the author of Pion! We looked at Pion and Janus when we were evaluating, but ended up using GStreamer ultimately because we had other GStreamer components within our pipeline.

That makes sense :)

If you saw anything when evaluating Pion that could have made it better I would love to hear. Always trying to improve. These threads are super exciting since I get a general guidance.

Existing Pion users are biased/accepting of its flaws ;)

I wanna believe that this is a fantastic piece of software. However it's really bad at describing what it does. As a software developer, I cannot find a high-level 2-5 sentence description of what GStreamer actually is/does on their website. I might just be blind, but this should really be on the home page above the fold. So many OSS projects fail at this :(

Its a multimedia framework. It can do everything in that domain, decoding, muxing, rendering, etc.

Their homepage does just get straight to the release notes, but if you select documentation on the sidebar you get a nice synopsis.

https://gstreamer.freedesktop.org/documentation/

I just deleted my reply to your comment, because I checked their homepage a second time and then saw this at the bottom:

> What is GStreamer?

> GStreamer is a library for constructing graphs of media-handling components. The applications it supports range from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.

> Applications can take advantage of advances in codec and filter technology transparently. Developers can add new codecs and filters by writing a simple plugin with a clean, generic interface. Read more ...

> GStreamer is released under the LGPL. The 1.x series is API and ABI stable and supersedes the previous stable 0.10 series. Both can be installed in parallel.

That should be at the top of the page.

I agree. Apparently it's just a library, so you have to use it to construct an application.

But they should provide some links to applications built with it.

It depends what you are trying to do much like ffmpeg they a command line tool called gst-launch-1.0 that can be used to do a lot (but I find the pipeline string syntax much nicer than ffmpeg's command line options), but you can also use gstreamer as a library embedded in an application (we've been very satisfied at my work using the rust bindings). One thing that is particularly nice is that the same pipeline syntax can be used embedded in an application or from the command line. If you want they some example pipelines here https://gstreamer.freedesktop.org/documentation/tools/gst-la...

That’s because it basically does everything, and is extensible via plugins below :)

I understand it seems hard to describe, because it is very versatile and does a lot of things. But not describing it isn't helping. Even such a broad framework can be described in a couple of sentences. As mentioned above, they have it in their "Docs", but not on the home page.

WebRTC has so many great implementations now! The author of the C# implementation started a really great project webrtc-echoes[0] that shows them all working together.

The next big challenge in the space seems to be getting widely available congestion control. The hard part is making sure it is understandable and customizable for everyones use cases.

[0] https://github.com/sipsorcery/webrtc-echoes

I am probably asking about an apples to oranges comparison, but could someone knowledgeable help me understand in what situations I would want to use gstreamer vs ffmpeg vs pipewire vs others. They all have some overlap and can use each other for some sub-tasks, but I do not know when to reach for what.

GStreamer is the top-level framework that allows you to connect the modules it offers to solve your problem, ffmpeg is a tool or library to invoke which may cover your case but is a bit harder to integrate and fit for arbitrary usecases, PipeWire is for live connecting input sources to output sinks at a lower level.

> ffmpeg is a tool or library

If you use the ffmpeg library, I will just say it's extremely unwieldy and difficult to use, and has barely any official documentation or examples. GStreamer is the opposite IME.

https://github.com/FFmpeg/FFmpeg/tree/master/doc/examples ?

In any case, they generally fill different niches. FFmpeg is, generally, a library containing set of A/V codecs and (de)muxers. (There are some more obscure parts, too, plus a command line tool ffmpeg(1) that gives access to most aspects of that library.) GStreamer is, generally, an A/V graph framework originally modelled after Microsoft DirectShow.

If you find FFmpeg too difficult to use, most likely what you want isn't a codec library in the first place, but some higher-level abstraction (e.g. if you just want a video on screen, most likely you don't want to muck around with low-level demuxing, but you'd rather want something that deals with clocks and sync and stuff, and FFmpeg just doesn't do that—it's not a video player). It has a lot of weaknesses, but the API is fairly straightforward for what it does.

it really depends on what you're trying to do.

gstreamer and ffmpeg can both fill the same niche, but not completely. gstreamer is plugin based while ffmpeg is monolithic. gstreamer has a plugin for ffmpeg, while the reverse is (or wasn't last i checked) isn't true.

You need pipewire, pulse, or plain alsa to actually play the audio streams. Not a lot of overlap most of the time with gstreamer or ffmpeg unless you're dealing with raw streams.

Well, one simple case is to make a media player. I used it to make a simple one[0] a few years ago (two bottom right windows, the top left windows are an unrelated directory browser).

[0] https://i.imgur.com/xSrUyhu.png

I would love to do buy gstreamer course to learn more, I am finding the documentation hard to navigate on the macOS platform, it seems very linux friendly. Keeping an eye for study materials.

Are you using their Application Development Manual? - https://gstreamer.freedesktop.org/documentation/application-...

There’s also a Plug-in Development Guide. Both of these are good resources. PDFs of the same should be available online.

I might switch to Epiphany on my Pinebook Pro due to gstreamer supporting hwaccel video decode. VAAPI (used by Firefox) does not on this hardware.

I'll have to run some tests though.

You can give Chromium a try. It has unofficially supported hw video decode capability. Though in my experience, out of the available browsers, Epiphany had the most robust hw video playback support due to gstreamer integration.

I am pretty sure Chromium does not have hwaccel decode on my hardware (rk3399). It uses VAAPI like Firefox.

My bad. I thought you were saying you had issues with getting vaapi to work on Firefox. Tough luck getting any hw video acceleration in Linux outside of x86_64 and vaapi capable chips.

These are incredible contributions to the community. Thank you Collabora and the gstreamer team.

GStreamer is so amazing! Probably the best way to do anything related to media.

Reply on news.ycombinator.com