timanglade

Assuming you had code that somehow could either be packaged as a linux container, or as a Wasm binary, then the advantage of the latter would be that yes Wasm supports multiple CPU architectures out of the box, it also consumes less resources (memory, etc.), will usually have faster start times, and the Wasm security sandboxing is stronger.

That’s not quite right. You can’t take an existing container app and just "export" it as Wasm. (Technically you might, but it would require a pretty big re-architecture and re-write, as Wasm doesn’t support garbage collection or multithreading at the moment. It also requires you use a language that can be compiled to Wasm, which can be limiting. Due to this, Wasm — at this stage — is probably best fitted to functions rather than full apps, although that is changing quickly.)

What you can however, is build apps for Wasm (or apps that combine Wasm and containers) with the same ease you currently enjoy when building pure container apps, i.e. see my comment above [0]

[0]: https://news.ycombinator.com/item?id=33324093

AWS announced support for containers on Lambda last year[0]

> the "main advantages" [redacted] describes are the same as Docker

Yup, that’s it! If you value Docker to build container apps, we think this will help you build Wasm apps in the same way, and the only container-centric abstraction this Technical Preview uses (packaging artifacts as OCI images) can be bypassed, if you prefer to deploy your artifacts as native Wasm binaries. The latter can be helpful if you are trying to get the full speed & efficiency benefits of a Wasm-native deployment, as the shim in our OCI package introduces a small performance penalty.

[0]: https://aws.amazon.com/blogs/containers/containerizing-lambd...

That’s our experience as well… This Technical Preview is an early downpayment, and we’re definitely looking for feedback on how one may could make the Wasm development experience better!

Basically yes: if you get value out of Docker for container apps today, we think you’ll get value out of Docker for Wasm apps tomorrow.

I tried to answer this above[0]. Instead of trying to explain it again, I’d encourage you to give it a try[1], and if after going through the 5-minute tutorial you still don’t get the point then a) maybe we messed up (and I’ll be sorry for having wasted your time!) or b) maybe it’s not for you (and I’ll also be sorry I wasted your time). It took me a while to wrap my head around this Docker+Wasm thing too when I first heard about it internally — then again it took me months to wrap my head around my first demo of Docker, so maybe I’m just dense!

[0]: https://news.ycombinator.com/item?id=33324093

[1]: https://docs.docker.com/desktop/wasm/

> I'm just sincerely confused at what problem this technical preview is intended to solve.

This is all good feedback, and we’ll definitely try to explain the added value better in the future. The main advantages we see in this technical preview are:

1. Easy, reproducible dev environment to quickly & reliably develop cloud/edge apps that target Wasm, or code frontend apps that target a Wasm backend (for example, as part of a microservice architecture). This is particularly helpful if you build apps that have a mix of Wasm & container components[0]

2. Easy way to share & deploy Wasm artifacts, using trusted infra like Docker Hub, but also Dockerfiles and Docker Compose

3. Transparent, reliable way to deploy Wasm applications to existing container-based infrastructure such as k8s (via OCI images) — but these apps can also be “unpacked” to run natively on edge infrastructure

> is it going to wrap the WASM binary produced by cargo in another layer of abstraction? If the latter, how is this new layer of abstraction different from just using one of the WasmEdge docker containers [0]?

Our approach is close to this. First, it was built with the WasmEdge folks, so you’re correct to detect the similarity. Second, it does wrap resulting artifacts into a OCI image, because we believe that can generate a lot of advantages (points #2 and #3 above) BUT you can also easily unpack the Wasm payload from the binary image at deploytime/runtime if you’d rather deploy your app on Wasm-native infrastructure (as opposed to container-native infra)

Appreciate the feedback. Hope the above helps.

[0]: https://docs.docker.com/desktop/wasm/#running-a-multi-servic...

> I don't know any dev looking for this or having a problem solved by it.

Judging by the reception at KubeCon & elsewhere today, we think at least some folks are excited by it. But it’s still early, and who knows, you may be right in the end. We launched this as a technical preview to test a hypothesis and learn from it, and so far the interactions from this HN thread alone have been greatly helpful.

"Docker+Wasm" is just a shorthand for the Technical Preview build, which allows you to build both traditional container apps, as well as Wasm apps. Behind the scenes, we try to let Wasm apps be developed largely without interference from any container technology — just giving you a good local environment you can use to code against. That said, if you want, we do offer the ability to run Wasm apps within a Docker Compose application. We do also offer the possibility to package Wasm apps within an OCI image, with an embedded Wasm runtime (WasmEdge) so you can a) easily share these via an image registry like Docker Hub, AWS ECR, etc. and b) easily run this anywhere you’d run a container. That said it’s not mandatory, and if you want the benefits of (a) without the benefits of (b) you can easily unpack the image to just get the Wasm payload and run that however you want. We dove into the details of the approach at Kubecon today, and the video should be coming out shortly.

Long live Wasm indeed! Obviously we feel a bit differently about Docker and k8s being in the past — Docker is used by 68% of professional developers according to the latest SO survey[0], and k8s is still growing in popularity at 28%. But obviously the technology landscape changes rapidly, and maybe one day (we hope) Wasm will be at 28%, 68% or higher. We’re frankly just excited about the possibilities, and wanted to help along the way :)

[0]: https://survey.stackoverflow.co/2022/#most-popular-technolog...

No disagreement here! Just to contextualize: Docker — the company as it exists today — is singularly focused on the development experience, i.e. the inner loop of code/test/build, not the outer loop of deploying to production (which is largely controlled by cloud platforms and k8s at this point). I know that separation can seem arbitrary, considering containers are largely successful because they help bridge the two, but that’s just the reality of what we’re focused on in our daily jobs @ Docker.

Within that framework, we see Wasm as extremely compatible with our goals of improving the local development experience, and yes, giving people alternatives to container-centric approaches. I’m personally inclined to agree with the points you are making about opportunities in orchestration, but we’re starting today by just trying to give people to a solid toolset that lets you iterate on your Wasm apps locally, and easily export the resulting artifacts, so you can deploy them as you see fit. In the process we try to be careful about shedding any container-centric assumptions, while porting over some of the wins of the docker tooling that we think can translate well to Wasm (easy local dev environment, standard artifacts, broad platform compatibility across Windows/Linux/M1, etc.) We will happily work with anyone interested in working with us to improve the production/deployment landscape for Wasm, and in fact I would say the main reason that drove us to launch this technical preview today, was to attract feedback on how the Wasm community (ourselves included) could best deliver an alternative path to production for applications going forward.

Hope this makes sense and that I understood your point accurately!

Broadly, we agree. The goal of this Technical Preview is not to encourage Wasm to be mediated by containers in production, but rather to enable people to locally build & package Wasm apps easily. In production, that could look like a number of different scenarios, from "bare metal" edge (just running in a Wasm VM), to running your Wasm workloads in a nomad/k8s cluster if that’s what you need (e.g. if you want hybrid container/wasm orchestration).

Ha seems like a fun coincidence. The writers came up with it early in the writing of season 4, and I started working on it sometime in the Summer of 2016 iirc. As far as the origin story goes, it was just great writers coming up with a great joke! Our lead technical consultant Todd mentioned to them we could actually build their joke for real, and the show jumped on the idea!

The amazing react-native-camera plugin! [0] I’m still getting a few camera-related crashes on Android right now, but overall I would say it makes things pretty smooth!

[0]: https://github.com/lwansbrough/react-native-camera

Great question — I did not, because I had unfortunately spent all of my data on that last training run, and I did not have a untainted dataset left to measure the impact of quantization on. (Just poor planning on my part really.)

It’s also my understanding at the moment that quantization does not help with inference speed or memory usage, which were my chief concerns. I was comfortable with the binary size (<20MB) that was being shipped and did not feel the need to save a few more MBs there. I was more worried about accuracy, and did not want to ship a quantized version of my network without being able to assess the impact.

Finally, it now seems that quantization may be best applied at training time rather than at shipping time, according to a recent paper by the University of Iowa & Snapchat [0], so I would probably want to bake that earlier into my design phase next time around.

[0]: https://arxiv.org/abs/1706.03912

While we’re here and chatting about this, I should say most of the credit for this app should really go towards the following people:

Mike Judge, Alec Berg, Clay Tarver, and all the awesome writers that actually came up with the concept: Meghan Pleticha (who wrote the episode), Adam Countee, Carrie Kemper, Dan O’Keefe (of Festivus fame), Chris Provenzano (who wrote the amazing “Hooli-con” episode this season), Graham Wagner, Shawn Boxee, Rachele Lynn & Andrew Law…

Todd Silverstein, Jonathan Dotan, Amy Solomon, Jim Klever-Weis and our awesome Transmedia Producer Lisa Schomas for shepherding it through and making it real!

Our kick-ass production designers Dorothy Street & Rich Toyon.

Meaghan, Dana, David, Jay, Jonathan and the entire crew at HBO that worked hard to get the app published (yay! we did it!)

My takeaway is that local development has a huge developer experience advantage when you are going through your initial network design / data wrangling phase. You can iterate quickly on labeling images, develop using all your favorite tools/IDEs, and dealing with the lack of official eGPU support is bearable. Efficiency-wise it’s not bad. As far as I could tell the bottleneck ended up being on the GPU, even on a 2016 MacBook Pro with Thunderbolt 2 and tons of data augmentation done on CPU. It’s also a very lengthy phase so it helps that’s it’s a lot cheaper than cloud.

When you get into the final, long training runs, I would say the developer experience advantages start to come down, and not having to deal with the freezes/crashes or other eGPU disadvantages (like keeping your laptop powered on in one place for an 80-hour run) makes moving to the cloud (or a dedicated machine) become very appealing indeed. You will also sometimes be able to parallelize your training in such a way that the cloud will be more time-efficient (if still not quite money-efficient). For Cloud, I had my best experience using Paperspace [0]. I’m very interested to give Google Cloud’s Machine Learning API a try.

If you’re pressed for money, you can’t do better than buying a top of the line GPU once every year or every other year, and putting it in an eGPU enclosure.

If you want the absolute best experience, I’d build a local desktop machine with 2–4 GPUs (so you can do multiple training runs in parallel while you design, or do a faster, parallelized run when you are finalizing).

Cloud does not quite totally make sense to me until the costs come down, unless you are 1) pressed for time and 2) will not be doing more than 1 machine learning training in your lifetime. Building your own local cluster becomes cost-efficient after 2 or 3 AI projects per year, I’d say.

[0]: https://www.paperspace.com/ml

Yes, that’s what you see in the picture, although as completely personal advice, I would stop short of recommending it. For one there are arguably better cases out there now, and you can sometimes build your own eGPU rig for less. Finally, the Mac software integration (with any eGPU) is very hacky at the moment despite the community’s best efforts, and I had to deal with a lot of kernel panics and graphics crashes, so overall I’m not sure I would recommend others attempt the same setup.

Well for a while I was lulled into complacency because the retrained networks would indicate 98%+ accuracy, but really that was just an artifact of my 49:1 nothotdog:hotdog image imbalance. When I started weighing proportionately, a lot of networks were measurably lower, although it’s obviously possible to get Inception of Vgg back to a “true” 98% accuracy given enough training time.

That would have beat what I ended up shipping, but the problem of course was the size of those networks. So really, if we’re comparing apples to apples, I’ll say none of the “small”, mobile-friendly neural nets (e.g. SqueezeNet, MobileNet) I tried to retrain did anywhere near as well as my DeepDog network trained from scratch. The training runs were really erratic and never really reached any sort of upper bound asymptotically as they should. I think this has to do with the fact that these very small networks contain data about a lot of ImageNet classes, and it’s very hard to tune what they should retain vs. what they should forget, so picking your learning rate (and possibly adjusting it on the fly) ends up being very critical. It’s like doing neurosurgery on a mouse vs. a human I guess — the brain is much smaller, but the blade says the same size :-/

Not offtopic at all! Dirty hack for sure. The enclosure I bought was a hack, the drivers were a hack, and there was software on top that was a hack as well. But the developer experience was totally awesome… Almost made the constant graphics crashes worth it. Almost.

Ha, I’d love to hear more. What was the app for? I can’t imagine why you’d have to pick up on Trebek’s elocution??

a gentleman never tells

SGD with Cyclical Learning Rates [0]. Honestly, it’s the closest to a Machine Learning silver bullet I’ve found to date! That paper is awesome.

[0]: https://arxiv.org/abs/1506.01186

Ha you have no idea how hard chicago hotdogs made my life! There was a joke in the show about Dinesh having to stare at a lot of “adult” imagery for days on end to tune his AI, but my waterloo was chicago hotdogs — the stupid pickles end up hiding the sausage more times than not, which makes it hard to differentiate them from say, a sandwich.

For those of you like me that never knew they existed, here is what they look like: http://img.food.com/img/recipes/25/97/2/large/picMKCD0m.jpg

:D now, that would be an app worth investing in

Ooops, good catch — I had posted the wrong definition. Corrected now! It was convolution, batch norm, then elu activation.

Thanks for the kind words! To prevent impostor syndrome, I should clarify that I worked on the app for many, many months — basically since August of last year — as a nights/weekends thing. It’s true that the final version was built almost from scratch in a few weeks, but it wouldn’t have been possible without the time investment in the preceding months. Although for the most part I just wasted a lot of time because I had no idea what I was doing lol (still don’t)

That’s not in the cards, at least at the moment, although if we get enough requests for it, I may be able to convince the powers that be…

In the meantime, iff there are any details you’d like to see, don’t hesitate to chime in and I’ll try to respond with details!

For sure, although I should say, for this specific instance I ended up training a network from scratch. I did get inspiration from the MobileNets architecture, but I did not keep any of the weights from their ImageNet training. That was shockingly affordable to do even on my very limited setup, and the results were better than what I could do with a retraining (mostly has to do with how finicky small networks can be when it comes to retraining).

Not at the moment — although you’ll find the most critical aspects explained in detail in the post. The rest I fear will age very quickly… With stuff like CoreML and TensorFlow Lite on the immediate horizon, I can’t imagine people will want or need to use the cumbersome manual approach I had to use to ship this app. Anything in particular you’d like to see? I can try to share it in a follow-up post or in comments here.