Realtime encoding - over 150x faster
transloadit.com
transloadit.com
A guy can dream, can't he?
At least this time the logo at the top links back to the main product page. Oh, and I see that the title text of the logo has just such a description! You're almost there guys!
This makes me sad. :(
"Since our servers can encode video much faster than most of your users can upload it, this means there is literally no more delay between the end of the upload and the video finishing encoding. In the screencast above this makes a 150x speed difference."
Surely the upper bound is 2x if you could transcode faster than the upload before.
Unless you are just measuring the time between the upload finishing and the transcode being done. But why would a user care about that metric rather than the total elapsed time?
That is what we are measuring indeed.
> But why would a user care about that metric rather than the total elapsed time?
Because that is the time where the user feels he has already done his part, but the site he is on is taking forever to do what he needs it to do.
Another reason why we measure the time from upload done to encoding done is because that's what our customers pay us to do.
So at some point in the future we'll offer upload servers in all major geographic areas.
Comcast used to route my traffic to California through Seattle, Washington, then down to San Jose and then Fremont, but now, it's going to Texas first, then across to San Diego, then up to Fremont, across a saturated link.
However, it wouldn't be particularly useful for things like queuing uploads from a directory. The point of NaCl is running native machine code in a provably secure sandbox. It's about making possible web app features faster on the client side, rather than adding new capability, per se.
If you start to think about input and output in web app as streams rather than buffered data, alot of neat possibilities arise for reducing latency.
However they do examine the buffer during an upload. Try uploading an MOV with bad edit points, they will warn you about A/V sync issues before the upload is complete. I assume they also abort uploads for invalid files.
How much work have you had to do around these issues? Did they prove to be bigger or smaller than you anticipated?
How do you deal with the licensing issues regarding open source encoding software? Do you pay the MPEG-LA a fee directly for use of the software? Is it per file/per minute/flat fee?
Just wondering about the mechanics of this, mainly for an in-house streaming application.
MPEG video stuff is generally pretty cheap or free for low volume. Audio codecs are fairly expensive, on the other hand. AAC + MP3 + AMR is >$20K in minimum fees.
So sure, there could be problems we don't know about yet, but that's the price for doing something nobody has done before. We will deal with these problems as they come up. In the worst case that might mean falling back to normal encoding if a ffmpeg child process starts to misbehave.
Ffmpeg only reads as much from stdin as it can process. Node.js gives us a 'drain' event when this happend, which means we can pump more data into it. So we are no throttling the speed, just the flow of data.
Ultimately, if you are a provider of services, these or any other, you will add functionality or run the risk of being eclipsed.
Also, we don't believe individual features themselves will give us a long term advantage. Consistently being a leader of innovation however, will.
2. Publicity. There are a bunch of features that would certainly be equally useful to our customers. However, none of them would get much interest from hacker news. We got a lot of signups and page views from this.
3. It took me probably 4 full days [2] to implement this, including the screencast and blog post. That's reasonable for the value added by this feature.
--fg
[1] We are really infinitely faster if the upload speed is slower than our encoding speed. This is probably the case for 99% of all consumer internet connections at this point. In any case we reduced the time required for an encoding from SUM(Upload, encoding) to MAX(Upload, Encoding).
[2] We already made a lot of technology decisions that lowered the cost of implementing this. A competitor would certainly need much more resources to pull this off.
I've never used your service so I'm not sure exactly how the upload stream is redirected to your platform, so this concern might not be totally valid if the upload is running through the client platform anyways.
If the upload doesn't finish the user needs to redo it.
> It seems one advantage of the two step process would be the ability to try the process again
You got me confused here. If the upload never finished, there is nothing you can do to fix this.
That being said, resumable file uploading is the next thing we'll tackle.
I should have been more specific: consider the special case that the encoder on your side encounters a random error whereas just a "plain" upload to the client platform would have finished, so precluding any network transmission errors.
Although, with resumable uploading and a simple go-between service on the client platform between the user and your platform all manner of fanciness could be achieved, I think. Thanks for quick response!
Maybe it's not ready but it seems to be something to look forward to.
Why would I want to upload 4GB of video when I can encode it down to 700MB then upload?
The pricing is a tad expensive, but doesn't look bad at all when you consider the need for an on demand encoder for the entire time that the user is uploading.
If you have a project and pricing is a deal breaker, just email us and we'll set you up with a discount.
Why is it difficult on most stacks? Because it's tying up a request handling thread?
Sure you can do this with threads, but it's gonna get very tricky to pump data between a socket, file and process using threaded programming.
However, I don't see why it'd be tricky in a threaded server. The thread gets an fd to recv() the incoming data from, and it popen()s an ffmpeg process then loops to recv() and write() the data until done.
But most people just sit on a request/response oriented Python/Ruby/PHP stack with possibly stupid buffering load balancing in between (nginx buffers uploads).
If you build this from ground up, it is certainly possible with a lot of technologies.
edit: I usually don't bitch about being downmodded. But don't you guys know any old people? And know that old people tend to be in charge of things? In any event, you shouldn't rely upon business leaders of any age being ignorant of the language.
-http://www.thefreedictionary.com/sucks 5. Vulgar Slang To perform fellatio on.
-suck, Old English sucan, corresponding to Latine sugere "to suck." It's of imitative origin. Meaning "do fellatio" is first recorded 1928.
-Slang sense of "be contemptible" first attested 1971
Honestly?
You see the word "sucks", in the context of something not being very good, everywhere. Literally everywhere. I bet that if you googled any tech-related keyword, somewhere in the first ten results, "sucks" would be employed in the context of "is not so good".
My point is this: People that read that blog post _will_ know what "sucks" means in that context and will not infer anything sexual from it - and even if they did, it's hardly the end of the world. Surely nobody who spends any reasonable amount of time reading blogs about video encoding will be in the slightest offended by fellatio. After all, "old people" are no different to "young people" except they have more experiences under their belt (hyuk hyuk).