Hacker News Reader

Comment by stevepike | Hacker News Reader

(founder here). The interaction between high level language package managers and system binaries or compiled extensions is definitely on our roadmap. For example the ruby net-ssh gem relies on specific versions of openssl being compiled on the machine. Do you think this is particularly difficult in ML?

Is your frustration mostly from needing compile all this non-python stuff across environments?

View on HN

Yeah, you're right on the money. This is basically insane in ML, but _especially_ in corporate environments, think F500, where sudo is disabled in containers and experimentation (in terms of trying different package versions) is quite challenging due to security restrictions.

Frankly, it's one of the reasons that we're seeing a lot of excitement around stuff written in Rust, but with Python bindings[1][2]. Cargo makes it so easy to build across various platforms and then just produce statically compiled platform specific python wheels. If PyTorch and TF had been written using Rust, any other other ultra high perf language with a tool-chain as reliable as cargo, a lot of folks lives would be easier.

[1]: https://www.getdaft.io/

[2]: https://www.pola.rs/

Hmm, the Debian package of ruby-net-ssh doesn't even depend on OpenSSL at all.

Interesting. Should it? I don't have much experience installing ruby gems with the OS package managers, I've always done it through the language-specific ones like bundler / gem. Here's a github issue showing the kind of things that come up when the versions are mismatched: https://github.com/net-ssh/net-ssh/issues/843

I was probably looking at an older or newer version to you, likely it switched from OpenSSL to pure-Ruby implementations of the crypto stuff.

Reply on news.ycombinator.com