DIY Video Hosting
tyler.io
tyler.io
This manifests as buffering on slower connections, where YouTube or Vimeo would just downgrade the user to a lower bitrate transparently (or nearly transparently).
An end user today will expect that behavior and have very little tolerance for buffering if their connection is unable to smoothly play the one bitrate the creator published (no matter how fast the CDN is).
Edit: I’m aware that it’s possible to do adaptive bitrate streaming outside of using Vimeo or YouTube (as several commenters have explained below) however this isn’t what TFA describes and I think it’s important to note this deficiency in the author’s described approach to serving their own video.
[0]: https://github.com/vincentbernat/video2hls
[1]: https://ryanparman.com/posts/2018/serving-bandwidth-friendly...
[0]: https://blog.metamorphium.com/2020/07/20/diy-video-streaming...
My solution was pretty janky, but it's an interesting problem space to explore.
I think you're thinking of certain DRM mechanisms. HLS with FairPlay wouldn't work on an Android device running Chrome, just as iOS Safari doesn't support Widevine. But for most of us, we don't need DRM.
Firefox, Chrome, Edge, IE, do not support HLS.
In any event, nothing natively supports MPEG-DASH. https://caniuse.com/#feat=mpeg-dash Though there is a disclaimer "DASH can be used with a JavaScript library in browsers that doesn't support it natively as long as they support Media Source Extensions." the same is true for HLS, without the requirement of MSEs.
[0]: https://github.com/video-dev/hls.js/
[1]: https://developer.mozilla.org/en-US/docs/Web/API/Media_Sourc...
As a user, I _never_ want to watch shit quality content and _MUCH_ prefer buffering to it.
Back in the day, you could open a tab, hit pause, go back to whatever you are reading, and by the time you finish that article and maybe grab a cup of coffee, the video is fully loaded and you can watch at your leisure.
These days, all the websites are so smart that they realise that you are not on the tab and load nothing so even though the page is been open for 20-minutes, you still have the lovely privilege of sitting there staring at the throbber every so often, thanks to your work insisting on full VPN tunnelling.
Do work in a VM on your main machine. That way you can have full VPN tunnelling in the VM but the host (and other VMs if you use them) use your normal direct connectivity.
Of course you might be breaking rules in a way that might invite discipline, if working in a highly regulated environment where so measures are essentially dictated by your work's clients, by taking technical measures to circumvent policy, so take care.
BunnyCDN which the author is recommending does support the HLS protocol. What isn't stated is if there's any specific steps to achieve that.
You're right though, the author didn't factor this as a requirement to their needs.
I've turned off too many 720p-only talks because it's unreadable.
Which is... a bit shocking. From the title of the article, I assumed the content would be related to hosting video. HLS was the first thing that came to mind and I assumed it'd be mentioned up top. I clicked to see if it listed any HTTP-related considerations I wasn't aware of in addition to HLS.
Turns out it's just an advertisement for a (very non-video-specific) CDN service, with some lines tacked on at the end telling you to use Handbrake (oddly, the cli, rather tham ffmpeg???)
And it spawned a great discussion about self-hosting.
Highly (or highishly) technical people ready to pay for video courses. One can infer from that a they have a good enough connection.
Is not like everybody should be Facebook, and have his video content available for all devices and bit rates.
Youtube is pretty decent for that, you can either let it figure out what format to use, or force the resolution. That's a good compromise IMO.
I agree that here auto-bandwidth adjustment is the wrong answer here and would not be appreciated by the author's paying customers.
This may be generally true, but for any screencast (including the author's use cases), automatically downgrading to a lower quality video based on connection speed is a bug, not a feature.
Alternatively, is there a way to encrypt something, such that you can apply a transform to the encrypted data to downsample it without knowing the contents of the file?
The average end user maybe. Not all endusers - for most videos I would rather wait than get a lower quality.
You can also go one step further and deliver youtube video through an entirely custom player: https://github.com/videojs/videojs-youtube
For some people there are maybe many different reasons no to go with Google. But the reasons the author put up, are no reason to go with Youtube. It is the cheapest way to host video and then to embed it into your site.
https://www.blender.org/media-exposure/youtube-blocks-blende...
That hasn't been possible in some time (yes there are some hacky ways to get around it), and caused a lot of companies in my extended network to ditch Youtube for embeded videos on their website.
1 or 2 ended up with Vidyard (or a similar for-marketers solution I don't remember the name of). For the rest I don't know what they ended up with.
The risk is low, but if it's your livelihood, you would be foolish not to consider the issue. Just like paypal: everything is fine until it's not, and then you are stuck with absolutely no avenue of appeal.
For our business I guess it’s still worth paying, but worth being aware that not all unlimited is really so.
Mux doesn't have this feature (yet). The best you could do is sign every video and have it expire after X time. This can get quite complicated and is less secure since it's only restricted by time (IP address isn't an option either), but due to the nature of how it works, you need to make the video accessible for a bit longer than the duration of it.
If Mux were to add domain restricted security features, I think I would switch to them because having the option of using a custom player is nice for niche content.
https://github.com/ytdl-org/youtube-dl/issues/10541
Our own videos have been pirated already and are shared around some torrents sites and on telegram, and there's nothing we can do besides plead to people's sense of moral duty. We're not a huge Hollywood studio or a megacorp. We're just a small bootstrapped company and producing our videos is the most expensive and time-consuming part of our platform. Luckily we still offer some added value besides videos, and make it easy to "do the right thing" and pay a subscription. Yet, we just have to live with piracy I guess. Vimeo didn't prevent it, although perhaps made it only slightly harder.
It looks like the "hack" is to only set the referrer header of the request to be the domain that's whitelisted which can be done with any HTTP client, but the issue is also 4 years old. I wonder if that's been addressed because the referrer header is really not something you can trust from the client.
I wonder if this also means if you use browser extensions that focus on security (such as randomizing your user agent and deleting the referrer header) that those folks wouldn't be able to even watch the videos.
Thanks for bringing this to my attention because if domain restriction does indeed still work like this then I don't think I will use Vimeo for that feature alone.
Signed URLs with both time and IP address based restrictions would be ideal. I wonder if IP address based restrictions is something Mux has planned.
It's still a fairly effective protection for casual copying or link sharing, but of course it's easy to circumvent. Even DRM gets broken all the time, so it seems like a bit of a futile effort to try to prevent it completely.
I have no troubles serving 4K video (with a 720p low-bitrate transcode) with a simple <video> tag, and I do not pay for bandwidth. This month, I've egressed 11TB for $20, and I could easily triple that at no additional cost.
I can stream 4K without buffering fine and I've never had a user complain, so it works.
I see on their website you can upgrade to 10gbit metered for an extra $2/month but I don't know what they'd charge for the unmetered options now.
There are lots and lots of providers if you are happy with 1gbit unmetered though: I'm a happy customer of BuyVM (starting from less than $2 a month!) and Scaleway (bandwidth depends on your plan).
If you are happy to spend a bit more for your own dedicated box, OVH, Hertzner, Online.net are great options. All of them offer unmetered BW I believe.
I don't remember if this is on a dedicated box or a VPS, but IIRC it was dedicated and the VPS has a 3 TB/mo limit.
Your vps is perfectly fine so long as you're not serving traffic in China, India, etc. If you need to push data globally, need to handle very peaky traffic above the throughput of one machine, or need to have 99.9999% uptime, a cdn will look attractive.
An analog for me is car repair. I could take my car in to get oil changes, have brakes serviced, etc. etc. Disregarding the money I save the reason I don't use those services is because those technicians and companies do not care about the quality of their work on my property as much as I do.
As an example, I was feeling too lazy to do a radiator replacement myself and so I took it to a big shop. Terrible mistake, when I got the car back they'd unplugged some of the sparkplug wires and forgot to plug one back in. There's no way they didn't notice the rough idling when moving it out of the bay. Even worse they wanted to say nothing was wrong until I had them pop the hood and pointed out the spark plug boot just sitting there.
Edit: To expand on this, the answer isn't buy a CDN solution. The answer is make CDN solutions so easy to implement that anyone can setup their own with their own hardware hosted in colos around the world.
... update their CNAMEs to point at a different company's CDN or a different video serving platform, and wait for the DNS to propagate
To me at least, the risk of a CDN or a video hosting service going bad or pulling your content is mostly the same risk as "colos around the world" going bad or pulling your content. Possibly a lower risk for most people - the chance of Cloudfront/Cloudflare/Akamai or AWS/Azure/GCE going broke or shutting you down is quite a lot lower than the risk of other smaller colo providers and your own hardware going dark. There's probably a smaller gap between smaller CDNs and cloud compute providers compared to rolling your own - but for most use cases I'd guess having a "cloud agnostic' design and ability to switch compute and CDN resources quickly and easily is a lower effort and more reliable way to reduce existential risk to your business.
I'm sure there are businesses who's risk estimates don't match mine - Darknet Markets for sure, probably "adult content" as well (given the observed puritanical zeal some of that industry gets deplatformed with). But for most clients and projects I've worked on in the 15-ish years since "Cloud services" became mainstream, owning hardware and leasing colo space would have been a totally misplaced cost/risk optimisation.(And I say that as someone wo used to do 2 or 3 trips a year from Sydney to colo facilities in the US and the UK with my luggage filled up with pre-loaded harddrives and toolkits to screw them into servers we owned on both US coasts and in London. I kinda miss the travel a bit, but I _totally_ prefer being able to get my phone out and provision new cloud "hardware" instead of hoping i had enough hot spare capacity somewhere, and ordering new hardware and plane tickets to fix disasters...)
Building your own infrastructure is like living "off the grid" with a wind turbine and a well and a composting toilet. You might have mitigated some risks but let's not pretend that there isn't an enormous cost in doing so.
And your infrastructure still isn't really literally "off the grid" of course. It's dependent on colos, network providers, etc. Sure, there's far, far less of a single point of failure than using YouTube but you're always going to be dependent on others to some degree.
Further, no one has the time or the money to do everything ourselves. To the upstream comment, I have neither the time or the energy to take on car maintenance. I have other things to do even if it means I spend more money and sometimes the shop doesn't do an optimum job. I'm not going to criticize someone who wants to do things themselves, but you have to choose.
Has had more uptime than Cloudflare.
FYI:
"An existential risk is any risk that has the potential to eliminate all of humanity or, at the very least, kill large swaths of the global population, leaving the survivors without sufficient means to rebuild society to current standards of living."
Source: https://futureoflife.org/background/existential-risk
Also: https://en.wikipedia.org/wiki/Global_catastrophic_risk
> I could easily
> I can stream 4K without...
Of course that depends on your users too. You have one connection point to the wider Internet and differing peering arrangements might many some (or possibly many) can't access your resources that fast even if you can serve them that fast as far as your provider's egress points.
Those outrageously priced public clouds and CDNs likely have greater global peering arrangements that will vastly best a cheap VPS for a lot of uses. They also offer availability guarantees, bandwidth guarantees that a cheap VPS will never offer (that 10gbit bandwidth will be "up to" on a shared pipe), and other potentially significant considerations.
> I've never had a user complain, so it works.
Of course if all your users are in topologically convenient locations so they can pull content from that one VPS at a good rate, you don't see sufficient concurrent access (or don't at times other users of the providers' bandwidth are) such that the lack of bandwidth guarantees are not an issue, then you do have the right tool for your job. But this is far from a one-size-fits-all business.
(though of course if a user just goes elsewhere without complaining, how would you know?)
It's a pretty good solution, but not perfect. I have heard that some people have trouble watching content, most likely due to peering between Germany and the US.
There's no transcoding options for the end-users since I'm already hosting 1 TB of content and there's not really room to keep different versions. Mostly not a big deal for my users but it's probably not helping with the connectivity issues.
I've considered other options but there's really nothing that can beat this setup when it comes to price performance. Serving a terabyte of content for 25 euro a month is pretty unbeatable. I recently had a 24 hours period where the server pushed out like 1.06 TB of traffic. This kind of bandwidth would be killing my wallet otherwise. And since this is a non-profit project, I'm eating all of the costs.
I think that peertube will be another good option. You can self host it and there are also many available public instances of it. This is particularly useful if your userbase is very large because it uses Bittorrent(webtorrent) p2p to reduce load on servers and it increases availability of content if viewerbase is huge.
Regarding the seeding I was thinking that you could put a few dollars into seed boxes to bolster some older or rarely watched videos.
It wouldn't be hard to host, but you'd have to be confident in your service provider or consultant.
Of course I doubt Linus would ever be censored by Youtube, he was just an example.
Result: I don't use Vimeo.
Any alternative is better.
I don't have this problem with any other website (that I have noticed), including any video and streaming services.
I'd be curious if you get a captcha here: https://laracasts.com/series/laravel-6-from-scratch/episodes...
I'm using a VPN, maybe that's why. It's annoying as not only you need to fill the captcha, but it takes 30s to 1 minute for the video to load.
What happens if you don't use that VPN?
They seem to be very aggressive blocking IPs. For example, I have VPS on Digital Ocean which I use for small projects and also as a VPN server. The IP has been "mine" for almost a year, I don't use it to scrap sites or abuse any service... but I still get the damn captcha on their videos.
https://github.com/Chocobozzz/PeerTube
it uses web-torrent so you safe allot of bandwidth.
Webtorrent was just added to libtorrent, so hopefully it will be coming to standard torrent clients soon. I'm hoping this will lead to more seeders and wider adoption of webtorrent.
There is also the problem of seeding from battery-powered mobile devices being impractical. Hopefully that can be worked around somehow.
Let's say a video gets posted and gets a million views in the first day. That's a lot of load on a server. But if thousands of peers can serve each other the chunks, then the server isn't responsible for all of them.
So WebTorrent can provide a level of scalability without greater hardware. Just doesn't mean that most of the videos are going to be served by peers.
With the HTML5 video tag it's not much more complicated than serving an an <img>. You still "need" two formats, I know that sucks (but av1 will hopefully become the solution). A small to medium sized blog can just upload the files to shared hosting (or even archive.org if space is sparse). No JS, no CDN needed.
If you want a good video streaming experience for all your viewers, you need something called Adaptive Bitrate (ABR). Some protocols that you may have heard of the implement ABR include HTTP Live Streaming (HLS) and MPEG Dynamic Adaptive streaming over HTTP (DASH).
ABR allows the stream to adapt to the bandwidth capabilities of the viewer. If you don't do this, some users will experience buffering, and some will have to sacrifice on quality.
Unfortunately most browsers don't support an ABR format natively, so you need to use Media Source Extensions, and a more complex player to support these protocols in a browser.
Obviously it's not going to be YouTube/Netflix-level anyhow.
So if your income depends on people not rage-quitting your page, you'll have to provide something that satisfies their "needs", or more appropriately "desires".
Or you will just get a lower resolution and you can start watching in an instant.
Yeah that's a hard no.
Not everybody in this world has such a good internet connection. There are still enough people that would be happy to even have internet...
So, by doing this, you make sure your content can reach more people. Which, to me, seems to be the whole point of making and putting it online in the first place
User choice and smart defaults would go a long way.
I'd still try HLS but this may be good enough for v1, depending on what you're building.
I have Plex (like a self hosted Netflix) on my home-server and some other people are also using it and on many different devices and connection speeds. It has to transcode the video on the fly and cache it depending on the client to have a good experience. You can't just serve 4k video to everyone.
When dealing with more than 10s or 100s concurrent viewers the required bandwidth on your server will be high and putting a CDN in front may be required.
[0]: https://github.com/Dash-Industry-Forum/dash.js [1]: https://github.com/video-dev/hls.js
Let's be more concrete. How many people will you lose today from your audience if you just offer 720p? Hard numbers please!
I was referring to (small and medium sized) personal blogs and websites (like the one in this post), i.e. not a digital commodity on a market. If you're out there to make money I realize you have to be more fancy with what you put out there.
Around the 1:14 mark he goes into why you can't just use the html video tag. It was one of the best explanations I've heard around HLS and what it takes to stream video efficiently.
Streaming is more, storage is less.
No, no and no. You still don't host your video content yourself. You simply found another cheaper service and despite your claim this isn't some paid ad (quote: "This really isn't any sort of paid or sponsored content crap for Bunny."), it's exactly that.
See http://blog.bellebcooper.com/leaving-microblog.html for a good breakdown of two different viewpoints about how people think of ownership.
Whether they're paying someone else for infrastructure is hardly remarkable. Almost no one is completely off the grid. (Do you have your own power substation?) The author controls his own namespace, so he can swap it out any time and do so seamlessly. Contrast to Vimeo or YouTube where you have no control at this level.
Regarding Bunny, it may be a paid ad, but they do have very cheap prices. CDNs starting at 0.005/GB with pops in NA, Europe, and Asia are hard to find.
At what point would you call it self-hosting? Keeping it on his own servers? Rolling his own CDN? Bare-metal in a server farm?
> despite your claim this isn't some paid ad (quote: "This really isn't any sort of paid or sponsored content crap for Bunny."), it's exactly that.
I don't understand this comment. Do you have some sort of proof that the CDN reached out to the author, or even knows he exists outside of a CMS?
I've seen this sentimentality a lot, where people aren't allowed to talk about a service or a product positively without some sort of money being involved or sneaky guy-in-a-trenchcoat bs. I really don't understand it.
* He's doing IaaS instead of SaaS. Different layers.
He hosts the videos on S3, the CDN fetches the files from S3 and caches it. He's using the CDN for the cheap traffic/more pops. This would work without S3 and without a CDN too.
I'd pay them more if they handled adaptive bitrate automatically.
I would recommend BunnyCDN every time without hesitation.
Because I distinctly remember random iOS devices would not play certain files or fail to sync.
This is why you use YT or Vimeo and the like.
Just to be sure, offer at least one version of the video with VP8 just to cover weird cases where h264 isn't supported. You can also encode 480p, 720p, and 1080p versions for users on slow connections.
Now, will it load on the original iPhone or IE 6? Maybe not - they'll see a link to the video file instead of a player - but not everyone needs to support legacy devices and browsers.
YouTube, Vimeo, and other similar services are nice because they handle everything for you, but sometimes they become too expensive or are blocked. Depending on the type of user you have, <video> with a h264 video is cheaper and works well enough.
I guess OGV could be used for even better support, but it doesn't compress as well.
[0] https://developer.mozilla.org/en-US/docs/Web/Media/Formats/V...
So you could "migrate" existing apps to say, use some machine at a colo instead of a variety of s3 services without much effort and thus unshackle yourself from a vendor.
Maybe you are making gobs of money and don't care, but a lot of people aren't and saving, say, a few hundred a year sounds very tempting. Not every competent person is rich
https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619...
If it's equally hard and an equivalent amount of effort than starting from scratch then it's completely a non solution. It's just another giant mountain to climb.
I'm looking more into it. It looks very complex and sophisticated. It's also built on a cloud "marketplace" where you're still doing SAS, now just in a way more complicated way with some wonky exchange system like you're a municipal energy broker.
Apparently they even sell multi-tier training sessions to understand all the jargon and complexities. There's no pretense that it takes anything less than months or years of study to use the thing
No, this is not it at all.
I'm talking about a worry free, drop in, simple replacement, an hour max. Set it, forget it, walk away. Not a multi year career path.
I swapped out s3 with simply doing scp and holding the same interface. It worked fine.
Does it scale to a 100 million users? No of course not. The point isn't to provide giant scale solutions over multiple continents but something to replace a ~$20/month service that a few hundred people use and make it effectively $0.
There's a huge market for that and it's the vast majority of cloud software written.
People will defend these complexities as The Right and Proper Way except it isn't. I've built half a dozen companies for people using $10-$40/month VPSs that do 6-7 digit annual revenue and run for years. All this autoscaling kubernetes prometheus blah blah blah, totally unnecessary. We aren't serving terabytes per second of video here... Transactional RDBMS, a Crud interface, basic file store, done and done.
The problem is all these systems are designed to facilitate building the next Paris or Hong Kong and if you're just trying to build the equivalent to a store on the side of the road, you have to do it the Paris way. It's nonsense.
People have other things to do with their lives than spend vast amount of time keeping up with the latest versions of irrelevant stuff to solve problems they will never have.
I wonder if there's a way to build a database on top of a CDN, or a webapp backend (might just be something like serverless, but instead of function definitions you have files+config a CDN automatically knows how to run). Something like Plan9 for the web.
While the author's answer may not be perfect or precisely tailored to your situation, they cover all the bases IMHO. A must read.
And several of the comments add to the discussion.
[edit: ufind -> us find]
Most likely it's just to satisfy some sort of nerd-ocd.
On the other hand, there is a ROI on building things either way. Always tough to measure
Or any similar fast CDNs allowing for audio files.
I serve all kinds of files <10mb via CF, that includes audio, video, images etc.
Files over >10mb I handle myself. Around 100TB is handled by CF each month and around 700TB by my own servers (4 Hetzner dedicated servers).
The whole setup costs me around 300 euros/month paid to Hetzner, and 20 to CF. That means I pay around 40 cents / TB.
So far CF hasn't complained, and I've been using them for years. Though they try to pitch me their higher priced plans now and then.
They do have an optimized product for that with Cloudflare Streaming.
They do cache small video files, even for free, but I think they don't want you to run a streaming business on top of them (without using their dedicated service).
While I bought radio streaming licences in the past, my streams always got shut down on platforms like YouTube.
If a client or project's content is mainly intended to be viewed and/or discoverable via the project website, I'm happy enough to have a plan along the lines of "We'll publish/host the video on Youtube (under an account that's exclusively for it and definitely not in any way linkable to any important Adworfds, gSuite, or Android dev accounts), and if the shit hits that fan there and they shut it down, we'll make sure we can reupload the videos to Vimeo or DACast or S3/Cloudfront or wherever, and just flip the video.example.com CNAME to match."
If you're relying on Youtube/Vimeo/Instagram/whatever for _discovery_, then you've got a totally different problem. (And, as far as I can tell, one without any good mitigation strategy. If you build all your brandvalue in a youTube channel, you're totally at the mercy of shitty cotent monitoring algorithms, automated copyright or DMCA takedowns, and 4Chan brigade attacks...)
I would highly suggest keeping the same CDN as the cloud/storage offering you work within. You get dinged quite a bit on transfer costs otherwise. So, if you're on AWS, stick to Cloudfront, etc.
Potentially relevant to post:
Note: I am the founder of Loom, so this is obviously extremely biased.
With Loom (https://www.loom.com) you will soon be able to upload unlimited videos (as well as record since we have recorders) to a plan for $10/month. No caps on viewing or storage.