My two favourite (underused) HTML5 media features | Hacker News Reader

My two favourite (underused) HTML5 media features

daniemon.com

36 pointstagawa13 comments

For playbackRate attribute, following is interesting:

>The cool thing is the audio pitch stays the same so human voices don’t go higher or lower and sound silly.

What algorithm would achieve that?

This kind of feature can often be found on professional DJ equipment and is usually called "key lock" or "pitch lock". On Wikipedia its referred to as "Time Stretching" [1]

[1] http://en.m.wikipedia.org/wiki/Audio_time-scale/pitch_modifi...

It's called timestretching. You basically have to chop up the sound into little bits and play them back in a staggered fashion.

The technology has a history from before the digital age: http://www.wendycarlos.com/other/Eltro-1967/index.html

Boutique DSP developer Sean Costello has done a great series of articles here: https://valhalladsp.wordpress.com/2010/05/06/digital-pitch-s...

  I’d love to see this added to the many HTML media players
  out there because it’s so useful to be able to slow down
  complex educational explanations or speech where the
  listener is a non-native speaker. Similarly, it’s helpful
  to speed up media when you’re in a rush or simply bored.

I work in a research lab studying & developing software for teaching languages, and this is one of the major reasons we decided to focus on HTML5 playback for annotated videos[1], which we've currently got piloted in several college-level French and German classes. When watching / listening to lecture material in a language they are familiar with, students tend to fast-forward and consume the material between 1.5 and 2x faster than real-time speed; meanwhile, when watching difficult sections of dialog, they'll slow it down and listen over and over[2] to make sure they hear it right. Unfortunately, not all the resources we use are available in HTML5-compatible form (i.e., we don't have access to a file, we're just embedding a YouTube video or something), and so we provide a wrapper around a bunch of different services' embeddable players to make them all look the same to the student, but most of them don't support changing playback rate, and it's a sufficiently popular feature that we get bug reports saying it broke whenever it has to be disabled because of the underlying service not providing a suitable API.

[1] Real-world (i.e., not made specifically for pedagogical use) media featuring real natural language usage that's been marked up with captions, glosses, commentary, etc., to make it accessible to lower-level language learners. There are several interesting start-ups in this space right now, like MovieMouth, which produces auxiliary content for Disney films to teach English to Spanish speakers (you must provide the actual Disney film yourself).

[2] Which brings up another killer feature of HTML5 media: arbitrary seeking. Makes it really easy to implement auto-repeat, which is great both for students wanting to listen to a few seconds over and over, and for transcribers who need to produce the annotations we use. It is so freaking annoying when Brightcove or YouTube or whatever other player will jump around whole seconds before or after where the users tells them to seek, or just refuse to seek at all sometimes. Another major source of bug reports from students who get used to HTML5 just working.

This is awesome, both features that I wish would be available in all video players. The questions that naturally arise:

1) How does one expose all these features without UI overload? You need a widget for the playback speed, and a widget to turn auto-pause off (in case I want to play a song in a background tab). More thought on this is needed.

2) Can we retroactively add these features to HTML media everywhere through a browser extension? This might address problem 1 by letting you choose an extension that only exposes the features you personally are most interested in.

(Note: The auto-pause seems to work if switch to another tab or minimize the Chrome window, but not if you switch to another window (Chrome or not) by clicking the taskbar button. But hey, still better than nothing. And combined with the speaker icon shown on sound-emitting tabs, this could go a long way towards preventing that annoying situation of not knowing which tab has begun auto-playing in the background)

I totally agree about the risk of UI overload. You could add them to a settings dialog but then features risk being missed by non-techie users.

The extension idea is good and one I've been looking into for Page Visibility but again, it needs to be configurable for times when background playing is wanted. But as you say, better than nothing.

"You could add them to a settings dialog but then features risk being missed by non-techie users."

Well, at least in that case the addition is a strict improvement: some users will find it useful, and some won't, but no one will find it harmful. With any other solution that exposes the functionality, there's the possibility that some users will find it confusing or offputting.

Those are really awesome features! HTML5 is really great with these features such as geolocation, media, localStorage, camera and the great mobile APIs like tapping into sensors, vibrate, even battery, all with very simple javascript.

Haha, anything said at half-speed sounds hilarious.

Go to 1:20 of the OPs demo (http://daniemon.com/tech/html5/playbackRate/) and set the playback rate to 0.5, "It's like they're like time machines............ you can get from here to Montana in about 3 hours........... you can go as fast as you want........ hahahaha..... and nobody can say anything." Very similar to http://www.youtube.com/watch?v=jSHNyppwS5w

Funny you should pick those two, since if the page "helpfully" uses the visibility feature to pause a video that I'm not staring at, it doesn't matter that I was using playbackRate to listen to it at 2x speed because it'll be at 0x speed.

Well, the visibility API is only of very limited use. The document will report itself to be still shown even if the window is minimized or in the background (or on iOS if I switched to another App!). It only reports the document as hidden if you switch to another tab. So if you want to use it to e.g. pause animation if the window isn't shown you will still burn unnecessary CPU cycles (ok, on iOS any timer is paused automatically if you switch to another App, so it kinda works out there - but not on the Desktop).

The more people use it, the more likely it is that mobile browsers will optimize for power consumption by doing things like toggling visibility if you've registered a state change event handler.

It's not that it doesn't call the event handler. It's about in which situations it's called and in which it isn't. And because the situation is the same on iOS Safari, Desktop Firefox and Desktop Chrome I guess it is meant to be like that? Anyway, as I said because timers are paused for background apps it isn't really a problem on iOS, but it is on the Desktop (or Laptop).

> It's about in which situations it's called and in which it isn't

My point was that those situations are not fixed forever. It'd be very easy to imagine e.g. MobileSafari starting to trigger visibility hidden when you switch to another app.

Reply on news.ycombinator.com