Amazon Elastic Transcoder - Watermarking, Bit / Frame Rate Control | Hacker News Reader

Amazon Elastic Transcoder - Watermarking, Bit / Frame Rate Control

aws.typepad.com

35 pointsjeffbarr24 comments

I'd love for there to be an AWS photo transcoding service as well. If you have a large site, creating your own photo transcoder workflow is not a big deal, and you can make it elastic by combining a bunch of other AWS services, but for a small web site it's a different picture.

For me photo transcoding comes in batches that take 5-15 minutes to process, a few times per week. This work bogs down the tiny instance, so I have to keep around a medium instance just for the occasional conversion work, but 99% of the time it's a waste of hardware. It's also a waste of my time having to write code to deal with things like iPhone picture image rotation and so on.

Have you looked at PiCloud? It looks like exactly the base you want to build on. They run AWS instances, but charge by the millisecond. You can run one up just for the 5-15 minutes it takes.

Or if you want to get fancier, try their queues [0], they've got an example of doing image processing to explain how it works. Essentially, you set up your processing pipeline attached to a queue. You then drop messages on the queue whenever you want to do work, and they spin up one or many machines (controllable by you) to process the data. Then after 20 seconds of nothing new appearing on the queue, they turn them all off.

They'll even give you 20 hours of compute time free, every month.

Edit - if you don't want to build it, drop me a message, I can do it.

[0] http://blog.picloud.com/2013/04/03/introducing-queues-creati...

I'll spend some more time on it, but right now it looks like another platform I need to learn, which I don't have time for.

What I want is a service that takes this URL and returns a resized JPG:

  http://SimpleImageService.com/resize
  ?account=DenisM
  &source=http://s3.amazonaws.com/DenisMsPictures/Image1.TIFF
  &size=400x400

That's it. I'll drop this into my app and forget about it. Set the prices at your AWS cost + 25%, and send me the bill at the end of each month.

When I was working at Technicolor/Medinavi (after they acquired most of the chumby employees), I wrote basically this exact service (massively parallel with goroutines and with filesystem-based caching, and they were running it on an AWS instance) in about 100 lines of Go code using ImageMagick through CGO (because ImageMagick has a lot more image operation and resample filter options than the built-in go image/draw package).

Sadly, I obviously can't just share the code because I don't own the copyright.

That's exactly what Stupeflix Tasks will do in a few weeks, along with video thumbnails strips, thumbnailing of videos at any point in time, video uploads, audio waveform, text to speech, video reverse etc. The beta doc is at https://stupeflix-tasks-api.readthedocs.org/en/latest/versio... , feel free to give me some feedback on it ! And if you want to do way more than just transcode, but create completely your video from images / music / videos , you can use the video rendering service at http://developer.stupeflix.com/ . It will migrated to the Tasks api too.

(Disclaimer : I am the CTO )

Well I need a side project at the moment :) I'll try and put something together and drop you a message.

Cloudinary does that. We used it for a bit but ultimately decided to roll our own. Their service worked great though. We've tried a bunch of options since hosting on Heroku makes large scale image processing a chore. Cloudinary was best of the bunch.

Check out imgix.com

Photo transcoding is a fast enough operation that for most sites doing it on-demand with a cache of previous results makes the most sense.

Eg. store only the original to start, when 1024p wide version is requested, check filesystem or database blob cache for 1024p version, if it exists, serve it, if not, transcode then serve it, cache it so next time it is requested you'll have it. This way you don't spend cycles transcoding photos at resolutions that are subsequently never requested as various sizes.

And existing APIs like ImageMagick make all the normal image processing features a breeze to implement.

Granted, photo transcoding is still a good idea for a service for people who really don't care about it and just want something that works out of the box, but video transcoding is a much tougher nut to crack (despite ffmpeg/libav and other solutions, there's still a lot of black magic people who haven't gotten their hands dirty with this sort of thing don't realize is an issue until they start transcoding videos and noticing audio sync and other such issues) and thus I'd be way more likely to pay for a service to do it rather than roll my own with video as opposed to photos.

The big thing that's tripping us up with using Elastic Transcoder is that there's no way to see how much billed time each encoding job took. We factor our encoding time in to what we bill our customers.

They've been iterating quickly and we're so close to being able to use this, but the billing thing is a big hurdle for us.

I have passed this along to the team. They'll be watching this thread for additional feature requests...

Amazon Elastic Transcoder provides the video duration for transcoded jobs through the API and the AWS Management Console. See https://forums.aws.amazon.com/ann.jspa?annID=1958 for more info.

I don't know how I managed to miss that, but thanks for pointing this out. The bitrate and framerate settings plus this have us sitting pretty for using Elastic Transcoder now. Exciting stuff!

As an addendum for everyone else, the ET team has been super helpful, and is much more active in their support forums than most of the other AWS teams. These guys seem eager to earn your business, and have iterated quickly.

Big thanks goes to them for moving fast, getting involved, and acting [quickly] on user feedback.

If there's anyone out there watching, I would love it if there were a similar AWS service just for audio transcoding (mp3, flac, etc.). Obviously I don't know what all is happening behind the scenes, but it seems to me that a lot of the infrastructure would be reusable.

I would also love it if streaming transcoding were possible with this API.

Would this support streaming? What about multiple outputs?

A bunch of gamers have issues live streaming to youtube and twitch. Each requires special (different) encoding parameters for streaming, and youtube needs each resolution streamed separately. This, along with separate streams for twitch and youtube murders most people's bandwidth not to mention a home computer's encoding capacity.

Most of the time they end up choosing only one site and a specific resolution and miss out on a large portion of their fan base. (People have a wide range of quality preferences/bandwidth requirements, in addition to being on different sites.)

Someone should use this to build a super easy multi-output streaming transcoder for youtube / twitch.tv / etc.

Guess the FAQ answers my question: http://aws.amazon.com/elastictranscoder/faqs/#Do_you_support...

No.

> ...have issues live streaming to youtube and twitch. Each requires special (different) encoding parameters for streaming, and youtube needs each resolution streamed separately. This, along with separate streams for twitch and youtube...

This isn't a sales pitch so much as an offer to chat about it and see if we could help turn a service we provide into a product that meets this need.

For years we've provided this as a "private label" service to major media companies that need to provide one single broadcast quality live stream and have it delivered to thousands or hundreds of thousands of end users on Windows and OSX (RTMP), Android (RTSP), and iOS (HLS), in bitrates ranging from 300 kbps to 6 megabits, while also pushing the full array of bitrates to YouTube, Hulu, etc.

The problem is that this is not cheap. Pulling it off with low latency and across a full array of formats and protocols tends to require dedicated hardware (depends on the source material, talking heads easier than sports, for example). But our approach does cost significantly less than trying to use AWS instances, and if the community of broadcasters or viewers was large enough, it might make sense for us to open up this private label service to individuals.

Feel free to ping me if someone wants to talk about it.

If the team is reading this thread: we cannot find a transcoding service that will output to intermediary codecs (non interframe codecs). We need to take 4k video in photojpeg, red, and prores and at least turn that into different resolutions of photojpeg. The output videos need to be of sufficient quality to use to produce commercial video, though not necessarily at the full 4k (or higher) res.

I wonder what strategies will be used by video start-ups who seem to be using AWS EC2 to cope with unexpected competition from AWS. AWS Transcoder seems to be getting better and their rates are pretty aggressive.

I am obviously behind since I didn't know about this service, but does anyone know whether they offer any sort of audio watermarking similar to the video watermarking service?

By audio watermarking, do you mean corrupting the audio with periods of silence/tone/noise that repeat, or do you mean a forensic watermark for audio fingerprinting purposes?

I meant mixing in a looping track the length of the main track ("brought to you by Mr X's Audio Tool", "this is a SAMPLE...", etc)

Reply on news.ycombinator.com