Hacker News Reader

Comment by scrollaway | Hacker News Reader

Feels like just yesterday that "doing video" was the most efficient way to burn giant piles of money.

What changed? Compression got better, but it doesn't feel like bandwidth got cheaper. Is it all just because it's easier to spin up a video hosting infrastructure on some cloud stack?

View on HN

A general understanding on how to scale massive video platform is well understood now more than ever, but it still remains a very efficient way to burn cash. To give you an example:

I own an ISP and about 2 years ago, we got first GGC node (basically youtube CDN), they are provided free by Google to ISPs as long as you are willing to host it at your own expense and you meet a certain bandwidth requirement, I think it was 1gbps youtube traffic at that time. Anyways they send us free 3 fully stacked Dell R720, shipped it us from another country. To the best of my knowledge, each of them cost 5kUSD. Total throughout after saturating ~30 Gbps, we will be given 3 more of these. In our country, to the best of my knowledge, there are 40 ISPs who have these nodes. A least 5 of them have 2 full racks worth of these servers - all free paid for by Youtube/Google.

This is just one country. Investment still remains an insane task for video hosting.

Bandwidth in terms of IP transit purchased at major IX points is a lot cheaper now. Also it is possible to offload a lot of traffic by settlement free peering if you have the money and scale to build a network with POPs in about 8 to 12 major carrier ix points in the USA. If you are a huge content source downstream singlehomed eyeball ISPs are happy to peer with you.

What caused this change? Carrier-hotel POP infrastructure [both technical and legal/regulatory] set up to serve Netflix and Cloudflare having positive externalities?

One of many reasons: Huge, fat pipes inter city are a lot cheaper now. What you used to pay for one 10GbE transport circuit on somebody's dwdm system will now buy 100GbE.

Another reason, stuff is clustered close together now. Exampels being the datacenters in Hillsboro OR and Ashburn VA.

Video ads are the only thing that pay more than peanuts these days, right? Banner ads and display ads are effectively worthless.

“Survey-ads” are an option too.

There’s a couple of similar Mechanical Turk micro-task vendors but they’re selective about who they partner with.

Search and social command decent amounts as well but those are effectively controlled by 4 companies. For the wide internet, yes video ads are where the money is at.

In a nutshell: video advertising became a very lucrative thing, easily offsetting the costs of bandwidth.

I think anybody who was involved in video from 2005 onwards knew this would eventually happen, but this really came to fruition in the past 5 years.

A webm is actually smaller than the original gif, usually small enough to also pack in an mp3 track and still end up smaller than the original. Compression getting better is a huge part of this.

but webm doesn't work on iOS, right? that seems like a deal breaker for a viral image/video site.

Imgur has been using video files instead of GIFs for years already, they call it GIFV. Anything over 2MB is autoconverted to (IIRC) h264 encoded mp4 files. They just added the ability to include sound.

Bandwidth isn't so much the problem, but transcoding. It's extremely CPU intensive and until very recently you either had to roll your own infrastructure from metal up, or pay insane EC2 prices to transcode uploaded videos. Now AWS has Elastic Transcoder, and there are a multitude of smaller SaaS popping up providing the same.

Doing video is cheap. Doing video well and without scaling issues (especially) live is still quite expensive. Not as bad as it used to be but still, it can eat up quite a bit of money. You'd need some way to monetize it if you are not going to limit clip length. Alternatively, at an even larger scale the equation turns around and suddenly you can set up your own fiber network and get some support from ISPs because you reduce their transit costs if they get your fiber into their 'meet-me' rooms.

Doing video on a third party cloud environment would be prohibitively expensive.

It’s certainly easier than it was before, but video is also as hard as ever because of the many moving parts. To replicate YouTube (in terms of basic features, not scale), you need:

- robust transcoding that is reasonably fast, into a multitude of different formats to accommodate web and mobile (example problem: do you support vertical videos)

- a good CDN at the best cost you can. videos need to start almost immediately

- ability to snap thumbnails and other artifacts

- lots of storage

- a fast single page app

- content controls (because a lot of people upload porn)

- a good extensible player that supports fullscreen and VAST ads and whatever else

- a robust uploader that can reliably handle large chunked uploads

I remember a quote from Chad Hurley where he said the reason YouTube succeeded was because unlike its competitors, who were all offering similar services, it didn’t suck. Given this laundry list of must-haves, you can see why it was so easy to suck.

I would assume they feel they can force people to watch a video ad prior to viewing a video, which isn't true for normal image hosting.

Can’t insert ads into GIFs easily. On video it’s trivial.

They serve tons of "gifs" as video files already, anything labeled as a gifv is really an html5 video element with MP4 or WebM. It looks just as good and loads much faster.

You could put ads on that just as easily, but it'd be a huge dick move if the ads had sound.

Right, and the draw of GIFs is that they have no sound.

I’m pretty sure this move might kill Imgur, not that they had a business to begin with. Reddit already switched to an in house S3 bucket and CDN for UGC uploads a while ago, content will either age out with attrition or be backfilled by users.

Imgur is going to be quite the ArchiveTeam project.

reddit video is complete garbage though

I've found it perfectly serviceable. Especially when youtube is blocked by corporate policy.

What corporate policy blocks Youtube but not Reddit? :)

Companies concerned about bandwidth rather than content.

I used to work somewhere that banned radio streaming sites, until the local ISP became willing to upgrade the connection.

They're probably paying for some low-tier bandwidth. That's something that can always be changed.

But the real upside is that your uploads will live along with your reddit post. Amazing how reddit becomes a dead-link factory when you go back just one year.

I don't know what low-tier BW means (I own an ISP), I think they problem reddit video is that their video infrastructure is most likely not geographically distributed and also not have peering with major telcos, which gives you high latency and poor playback for people who lives far from reddit video servers. Maybe they will eventually fix it, not sure if they have that much money to burn on repeated memes and porn videos.

They use fastly, a quick DNS lookup shows that even without whoising the resulting IP address

  > dig v.redd.it +short
  
  reddit.map.fastly.net.
  151.101.17.140

Then something is wrong with their setup. It's just not me. I have seen many comments on Reddit on how horrible their video platform is, it takes forever to load.

I give it about 50-50 odds that doing this doesn't kill it but relegates it to irrelevancy, like Photobucket and similar.

Maybe I'm unlucky but in my experience the reddit video player is terrible and maybe 25% of the of the time just gives me a black box or the first frame and won't play.

It also has the caveat that if you want to share it with someone you have to send them a link to the reddit comment thread, which is not always desired or appropriate.

BTW I've had the same problem, the solution is to just click it twice (once to "pause" it and once to unpause it). For what it's worth I've only gotten this problem since chrome released their update that disables autoplay (https://blog.chromium.org/2018/03/chrome-66-beta-css-typed-o...).

I think the black box just means that the first frame is black

Most of the times I've had it are on iOS, and no amount of tapping or putting it in full screen mode where you can use the native video controls will make it load.

I wonder if they're trying to serve WebM or some other unsupported codec to iOS devices by accident, but I'd think that would have some kind of error indication instead of a black box.

I recently discovered that youtube-dl works on v.redd.it links, so you can download and share the vids directly.

But I'm with you 100%, their video and image hosting is super slow, and needs to be paused/unpaused a few times before loading or playing anything.

I absolutely hate that you can't link directly to the media on reddit's hosting service. If I share something with a friend, I usually am not doing it so they can read a forum of internet comments first. This is something that imgur hasn't done and will continue to be an important differentiator for me.

Almost every image I see on Reddit uses imgur, and some subreddits have an explicit ban on using Reddit image hosting for some reason.

what's the difficulty in splicing in some additional frames at upload or request-time? From what I vaguely recall of the format it's essentially a stream of stream of independent frames with some timing info, although there might be something about colour palette selection? I'm sure there could be something that makes it especially horrible, but curious what it is :-)

Isn't video advertising mostly handled at a higher layer by sticking some segments into the playlist/container rather than actually encoding it into a single video unit anyway?

Maybe w MJPEG.

GIF is only 256 colours, so the renderer optimises the living daylights out of the palette to make things look vaguely uncrappy. Any ad inserted would have to only use those colours (which are then very different from GIF to GIF). That's a tough sell for brand graphic designers. GIFs are also often short loops, breaking the loop with an ad, especially when inserted automatically, will break a high number, if not most, of the good GIFs in circulation. Doing anything dynamically also breaks caching which is probably undesirable when most of your business is pumping dumb bytes around the net.

Finally, you can't add a link. I guess it's still useful for brand advertising, but I suspect that's less valuable than the kind where you can directly track conversions.

You can use 256 colors per-frame, as well as using transparancy. Also, each frame is just a diff from the previous frame's content. So it's not unreasonable, just pointless when video formats are so good.

Imgur doesn't use gifs. They transcode to mp4 and webm.

They do still use .gif in some cases

>what's the difficulty in splicing in some additional frames at upload or request-time?

Are you aware of the CPU overhead and lag time involved with just "splicing in some additional frames" to a video file within the context of an HTTP request cycle?

obviously it's not going to be free, I was more curious what made it specifically 'more difficult than video'.

I'd hope there would be some opportunity for caching to make it a bit cheaper than every single request, but maybe with dynamic targeted or real-time ad bidding that wouldn't work so well.

Middle-out compression solved it, when ran on the new Internet(tm), sharing bandwidth of all mobile devices in existence.

Reply on news.ycombinator.com