AWS' Sponsorship of the Rust Project | Hacker News Reader

AWS' Sponsorship of the Rust Project

aws.amazon.com

304 pointsGolDDranks59 comments

This appears to be part of a larger announcement that they are now providing AWS promotional credits to open source projects.

Source: https://aws.amazon.com/blogs/opensource/aws-promotional-cred...

Credits are available for one year. Sorry, this new credit program just seems like another customer acquisition cost and not a long-term commitment to supporting open-source projects.

I read this a couple of times and I can't tell how they are sponsoring the Rust Project. Apparently Rust uses some AWS services, are they now getting these for free, is that the sponsorship?

Based on the postscript:

> ps – AWS credits are available for other open source projects!

it sounds like it

Yeah, free CI and web hosting. Those are probably the biggest costs in running the project, besides paying developers to work on it.

"We’re thrilled that AWS, which the Rust project has used for years, is helping to sponsor Rust’s infrastructure. This sponsorship enables Rust to sustainably host infrastructure on AWS to ship compiler artifacts, provide crates.io crate downloads, and house automation required to glue all our processes together. These services span a myriad of AWS offerings from CloudFront to EC2 to S3. Diversifying the sponsorship of the Rust project is also critical to its long-term success, and we’re excited that AWS is directly aiding this goal." https://aws.amazon.com/blogs/opensource/aws-promotional-cred...

CI for Rust can be VERY expensive, given the vast amount of CPU and memory the Rust compiler needs. Good for Amazon to handle this important task.

CI is not run on EC2, MSFT is covering that on Azure.

Running with sccache drastically reduces build times. I suspect rust is using it to accelerate builds with it's s3 backend. I'm using the Azure storage backend for my GitHub action builds. Even on simple projects (3k LOCs, dozen crates), it cuts more than 60% off the build time.

By the way, we don't actually run our CI on AWS. We use S3 to store caches and build artifacts, but all the builds run on Azure Pipelines at the moment.

So with this announcement, I assume you’ll be switching to running your CI AWS soon (since it’d be free)?

Microsoft is providing Azure pipelines to the Rust project free of charge. Switching CI is not trivial, in all cases.

Additionally, we just switched to Azure pipelines after evaluating a number of options.

* https://internals.rust-lang.org/t/which-ci-platform-should-r...

* https://internals.rust-lang.org/t/update-on-the-ci-investiga...

Microsoft is already sponsoring our CI on Azure Pipelines, so there would be no need to do that.

This comment explains, from the horse's mouth... https://www.reddit.com/r/rust/comments/dhrkit/aws_sponsorshi...

PS. The comment on the reddit thread is from one of the co-authors of the blog announcing the sponsorship.

Check out crater: https://github.com/rust-lang/crater

The rust devs recompile a huge chunk of the entire rust ecosystem to make sure new compiler versions don't break things.

I can't imagine something like this being even close to feasible in the C++ ecosystem; there is too little standardization of build tooling, too much brittle/broken code.

Still it's not cheap and Amazon is paying, bless their hearts and excess compute capacity when it's not Christmas.

C++ being a standard, you don't need to rebuild the world to develop your compiler.

That being said, there is nothing preventing something similar with C++.

In fact, that's pretty much what Linux distributions and BSDs do when upgrading the compiler, see for example https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240629

How it's standardized doesn't matter. And neither language 'needs' it. But it would be just as useful to C++ compiler devs as it is to Rust.

Hard to find a corporate sponsors list.. but from the RustConf list, looks like AWS, Google and MS/Azure are all Rust supporters. Not to mention MS looking into replacing some Windows internals with Rust.

I think Rust will see huge growth from best of breed WebAssembly tools to the really small/fast libraries available. I'm not that deep, but it's been an interesting learning curve. Hoping to find a couple things to actually work on in terms of projects this coming year with it.

Edit: I am curious if we'll see some Docker tooling move towards Rust, most of that is currently Go, though some of the wasm stuff might be a better direction vs. containers.

why would docker tooling move from go to rust? I mean rust is good and all, but go had its advantages too would there really be that much benefit?

It really depends, rust doesn't have the runtime overhead and can do lock-less approaches to in-memory data. For example, etcd, kubernetes (service) and some other systems may reduce overhead and improve performance with a rust implementation.

You don't replace millions of line of code of something working with a language that looks faster on paper.

This is true in general. Kubernetes was ported from Java to Go, though I have no idea how large it was when it happened... that said I agree with you, and it's more likely that a new tool would appear than any of these deciding to suddenly re-write the universe.

While I agree... I wouldn't be surprised to see etcd displaced with a lookalike written in Rust. Sometimes people make things because it's fun to reinvent wheels at times. How many distributed big-table-like solutions are there?

Aside, I do hope that the I/O options for some of the wasm runtimes (WASI) popping up can work with the SQLite developers to get a build that works inside wasm, but can actually use the exposed file system interfaces... Not that sqlite itself is the goal, but imho enough of a filesystem focused library that will expose most issues on that part of the wasm runtimes.

There was talk about Docker supporting wasm alongside containers:

https://mobile.twitter.com/solomonstre/status/11110049132223...

That's part of what I was referring to... Also in my own reply regarding getting sqlite working inside wasm (wasi being part of the fs interface spec). AFAIK, sqlite won't currently work in wasm against an underlying fs exposed.

There seems to be an uptick in open source related announcements from AWS as of late.

I can’t imagine why...

All of the AWS teams have to lock what is going into reInvent weeks before the event. I'm assuming there is some sort of cutoff happening soon, so lots of activity is happening now.

No love for Ruby again, the folklore of Ruby hatred within Google and Amazon is quite real.

Ruby is used super widely throughout Amazon; any hatred is likely well deserved.

Amazon uses Ruby a lot? I've never seen them talk about anything with Ruby at all. Can you give any pointers?

It's a lot of internal tooling, likely nothing interesting to you or worth talking too much about.

> nothing interesting to you

How do you know what's interesting to me?

An attempt at a polite way of saying "I'm not going to talk about it." Please take a hint instead of downvoting :)

Sounds more condescending and dismissive, rather than polite!

And I didn't downvote you - I can't possibly have done, can I, because you can't downvote a reply to your own comment on Hacker News.

My apologies then.

Telling someone "this isn't interesting to you" in place of "I don't want to talk about it" is definitely not polite!

I'd say it's rude in general to tell someone they won't find something interesting, and especially so when it's done as a way of disowning your own preference not to talk about it.

There's way too much being read into a statement which begins with "likely". There are plenty of things in life that are hard to explain in a way that doesn't lead to misunderstandings which are also not very consequential.

For example, when people ask what I do, I could say I work at an ISP, or that I'm a systems engineer (my title), or that I'm a system administrator (some of what I do), or that I'm a software developer (the rest of what I do), or any number of other things. Depending on how interested in it I think they will be, or how interested in explaining it I am, I might respond that it's likely not that interesting.

If the person asking actually wants to pursue it further, the polite thing to do would be to say "oh, I find it interesting, if you're willing to talk about it". If they responded "How do you know what's interesting to me?" I would take that as somewhat aggressive, and definitely wouldn't be interested in explaining further, depending on how I perceived their disposition.

Perhaps it's a cultural miscommunication.

That phrasing (at least without the "to you") is fine when you are genuinely open to saying more. It's not fine as a way to refuse to say more. So you're talking about a completely different context.

If they get aggressive and you then decide that you don't want to explain, that's fine. But that's not what happened here.

> But that's not what happened here.

Eh, I don't think you can say that definitively. I took it as WaxProlix not wanting to/being unwilling to talk about it because it's boring, and as they found it boring, they thought other people would likely (the word they used, which I think people are ignoring) find it boring as well.

I'd agree with you if it were just the first message. But they then explained exactly what they meant and why, and that's what I offered them feedback on.

They apologized to the person they said it to, and it all seems settled from my perspective.

This is Hacker News. Rest assured, we find it interesting.

I don’t think anyone is going to give you internal information, but think testing and infrastructure

It's used all over the place. Around the time I was leaving (2016) I was hearing of some major projects that were planning their rewrite to include a migration to Ruby.

Amazon rarely likes to talk about how things are built at the nuts and bolts level.

I can't quantify compared to other languages but anecdotally there's definitely parts of the company that use it heavily. There's currently 146 job postings specifically mentioning ruby on rails[1] and 3154 mentioning ruby.

[1] https://www.amazon.jobs/en-gb/search?base_query=ruby+on+Rail...

I don't know why people are being coy. This isn't some secret sauce.

Ruby is a frequent scripting language at Amazon. JRuby on Rails is (or was) a solid way to get a nice web app running for internal use without interfacing with Spring. The AWS SDK is pretty great for internal tools that need to interact with S3.

It honestly isn't super interesting, but it's a well-supported platform internally.

Definitely not going to disagree with usage being frequent, but I wouldn't say Ruby is well-supported internally with respect to IDE support

I'm not sure what IDE support Amazon is supposed to provide WRT Ruby. Does Amazon develop and provide excellent Python or Perl IDE support? Even the Coral plugins for JVM IDEs were pretty rough.

Mostly connecting the package cache so every module doesn't fail to import in an IDE

My understanding (from Steve Yegge's famous 2000s Amazon vs Google memo) is that Amazon teams can use any programming language/platform they want, as long as there is no way for other internal Amazon projects to access it except via public (internal) API. Thus lots of languages and platforms are used internally, including Ruby.

Ruby?! 2016 called and they want their criticism back.

Ruby isn't an approved language at Google, I haven't heard it being talked about almost at all.

Python gets some dislike, and there are a few significant projects trying to get off it. As a blanket statement I don't think dynamic languages work well at a company like Google.

> As a blanket statement I don't think dynamic languages work well at a company like Google.

Looks like AWS is echoing that sentiment:

     Performance. Rust is blazingly fast and memory-efficient: with no runtime or garbage collector, it can power performance-critical services

Author here—while many systems are performance-sensitive, many others aren't. Personally, I don't enjoy writing Python at work due to its lack of misuse resistance more than any concerns about performance (although a large-enough project will eventually care about performance _at some point_), but I have little issue writing smaller Python programs that only _I_ maintain.

Google has plenty of python.

Ruby even excluded from GSoC while python isn't.

I've never heard that folklore, and the Ruby team is more than welcome to apply for support.

We detached this subthread from https://news.ycombinator.com/item?id=21251756.

I don't see any reference to Ruby in the link (or the one that this subthread was attached from). What relevance does Ruby have to this topic?

have been cursing ruby since morning.. but as the moon came out.. i slowly changed my mind

it is all about tooling and dev support. ruby wasn't landed properly

Reply on news.ycombinator.com