Rust libraries that need to be written | Hacker News Reader

Rust libraries that need to be written

mail.mozilla.org

167 pointssteveklabnik58 comments

I don't work with Rust, but in general this is a very nice way to present a project's needs, in order to request contributors to help. I hope it works for them.

I also like the pragmatic approach, instead of saying "date APIs are hard, so we'll think until we've cracked it" they just explicitly plan on copying a popular one (the Joda-Time Java API, http://www.joda.org/joda-time/).

As of now, copying an API is illegal, no?

http://www.reuters.com/article/2014/05/09/us-oracle-google-r...

It's not illegal, it's just apparently impacted by copyright. Since Joda-Time is available under the Apache 2.0 license, that doesn't seem like an unsurmountable obstacle.

I'm not a legal scholar but it seems like Joda-Time's Apache 2.0 license can be rendered irrelevant if Joda copies API spellings from Oracle Java.

For example, Joda has a member function called .getYear(). Oracle java.util.date also has a .getYear(). At the time the author created Joda, the type of restrictive API copyright ruling that Oracle won wasn't part any programmer's mindset. If it was, maybe Joda would have called it .getAnnum() or .getOffsetfromZeroCE() or .getYYYYComponent().

If a company gets to release their API first with a .ToString() ... the next company has to call it something else such as .ReifyAsString() and then then next company after that has to name their API with yet another synonym such as .Stringify().

Continuing that scenario, could one even publish a blog post listing the complete 1-to-1 translation of Company A's API to Company B's API? Could one legally write a program that automatically converts one source code listing's API calls to another? Would those rosetta stone type of publications and dev tools be in violation of API copyrights?

Joda-Time doesn't copy 100% java.util.date but I don't know what the threshold is.

Also not a lawyer, but IIRC Oracle won on the grounds that the "structure, sequence and organization" of their API was a creative and copyrightable work, not just the odd descriptive name here and there.

One of the circuit court's stated reasons for ruling that the SSO holds for the Java API was because of the names:

  The evidence showed that Oracle had "unlimited options as to the selection
  and arrangement of the 7000 lines Google copied." Appellant Br. 50.  Using
  the district court's "java.lang.Math.max" example, Oracle explains that the
  developers could have called it any number of things, including
  "Math.maximum" or "Arith.larger."  This was not a situation where Oracle was
  selecting among preordained names and phrases to create its packages.  As
  the district court recognized, moreover, "the Android method and class names
  could have been different from the names of their counterparts in Java and
  still have worked." Copyrightability Decision, 872 F. Supp. 2d at 976.
  Because "alternative experessions [we]re available," there is no merger.
  See Atari, 975 F.2d at 840.

http://www.cafc.uscourts.gov/images/stories/opinions-orders/...

pp. 30-31.

This is a broader point than "it was because of their names." As I understand it, the court is saying that Google copied the whole package structure of Java, including a whole tree of associated names. It's not just that some names happened to match up, but that the whole structure of the Java libraries was considered a creative work and Google copied that whole creative work. The names are a component of that, but you can't point at a name and say "That was the copyrighted thing," because it was the aggregate API that the court considered copyrightable.

(Again, not a lawyer or legal advice. This is just my layman's understanding.)

Yeah, I think you're right. It's the wholesale copying of a large chunk of the Java API rather than just the name of one thing.

Tangentially, I really disagree with the ruling. First, we have to figure out where to draw the line: if you can copyright a big API, can you copyright a small one (like, say, Iterable)? Second, there's legal precedent for allowing partial copies in cases where it's necessary for interoperability (Sega v. Accolade). Finally, the appeal for Lotus v. Borland allowed for wholesale copy of a command hierarchy for user interfaces, which has parallels to application programming interfaces that I think any programmer would find obvious. My favorite quote from that case:

  [51] That the Lotus menu command hierarchy is a "method of operation" becomes
  clearer when one considers program compatibility. Under Lotus's theory, if a
  user uses several different programs, he or she must learn how to perform the
  same operation in a different way for each program used. For example, if the
  user wanted the computer to print material, then the user would have to learn
  not just one method of operating the computer such that it prints, but many
  different methods. We find this absurd. The fact that there may be many
  different ways to operate a computer program, or even many different ways to
  operate a computer program using a set of hierarchically arranged command
  terms, does not make the actual method of operation chosen copyrightable; it
  still functions as a method for operating the computer and as such is
  uncopyrightable.

http://www.law.cornell.edu/copyright/cases/49_F3d_807.htm

Anyway, I know you didn't ask for my opinion on the ruling but I just wanted to rant a little :) The circuit's ruling really pissed me off. Programming is hard enough without having to worry about whether or not our APIs are covered by an existing copyright.

To add on to what chc said: We should all remember, as technical people, that these laws are meant to be interpreted by humans, not machines. Even databases of information aren't copyrightable, much less the function names that a programer uses.

You clearly missed the fact that Oracle won at appeal and indeed the function names a programmer uses are protected by copyright.

I made a getYear function a long time ago, stop stealing my goddamn copyrighted code!

You clearly missed that fact that no judge, anywhere, would hold up names as why something was copyrightable. Instead, as my sibling comments say, what was found copyrightable was the "structure, sequence and organization" of their API. The court said that Google copied the whole package structure of Java, including a whole tree of names.

I'm not a legal scholar but it seems like Joda-Time's Apache 2.0 license can be rendered irrelevant if Joda copies API spellings from Oracle Java.

Yeah. Let's just stop writing code alltogether shall we? Together someone somewhere probably has our future code pre-copyrighted.

If you respect stupidity, stupidity wins. Don't let it go unchallenged.

>Joda-Time doesn't copy 100% java.util.date but I don't know what the threshold is.

I think they're safe, especially given that the process has actually come full circle now.

The new date time package in Java 8 (java.time) was developed under JSR310, whose spec lead is none other than the developer of Joda-Time, Stephen Colebourne [1]

For those that are interested, he wrote an interesting blog post way back in 2009 on why they chose to develop JSR-310 instead of using Joda as is [2], as well a more recent piece on the bits that didn't make the cut (and are thus now part of a separate project) [3].

[1] https://jcp.org/en/jsr/detail?id=310

[2] http://blog.joda.org/2009/11/why-jsr-310-isn-joda-time_4941....

[3] http://blog.joda.org/2014/02/new-project-threeten-extra-for-...

Could Mr. Colebourne have done this work on JSR310 without an IP assignment to Oracle?

A Rust port of JodaTime should definitely be based on Jon Skeet's NodaTime (the .NET port) which probably maps easier to rust because of the closer similarity of C# to Rust (value types etc).

Whereas i'd say it should definitely be based on Stephen Colebourne's ThreeTen, which is the successor to Joda-Time:

http://www.threeten.org/

http://blog.joda.org/2009/11/why-jsr-310-isn-joda-time_4941....

The language constructs might not be as close to Rust's, but the semantics benefit from years of additional experience and thought.

I come from an image and signal processing background so for me one of the most important library features is a) a good support of largish one and multidimensional arrays, b) support for FFT and c) some support for linear algebra.

At the moment the Rust team doesn't really have the resources to spare on directly targeting numerical computing, but they are definitely interested in creating a language that can be 'grown' to suit the needs of a diverse range of fields that demand a high level of performance from their languages. They would definitely welcome community involvement in regards to helping build an ecosystem around numerical computing.

If those are your big requirements, you may want to try Julia. Those are the most mature parts of the Julia standard library are very stable, not to mention fast. Steven Johnson, the author of FFTW, which is the FFT library wrote the Julia FFT code and is Julia core developer. I think that it's safe to say for doing FFTs and linear algebra, there's no better system out there.

> We've previously made the decision not to distribute any crypto with Rust at all, but this is probably not tenable since crypto is used everywhere.

This is pretty much why I haven't used Rust for any serious application development. Any HTTP or TCP connection I'd want to make would be over TLS and there are better options than importing c crypto libraries in Rust.

I get what they're trying to do, and the conservative nature is admirable but I just want to "get things done" and while Rust is nice not having crypto is as is mentioned not tenable for application developers.

Crypto is very critical. Many people do not want their application to send their critical data in the clear.

Did you read to the end of the paragraph where they say that they're going to implement crypto?

They're not insecure about being able to make a workable crypto api. They are aware that it's not a good idea to implement crypto in a high level language that's not been engineered for security yet.

Their solution is to wrap a proven C crypto library, which is probably what good Go and JVM languages do also (I wouldn't trust one that doesn't) and it's probably going to be ready within a month or two.

edit: just an aside, why would you terminate your SSL in Go/Java, it's good practice to have a proxy like nginx or haproxy do that for you.

> Did you read to the end of the paragraph where they say that they're going to implement crypto?

I did, they said it would take a very long time.

> Their solution is to wrap a proven C crypto library

This is good. I don't have a problem with that, because that's what you'd have to do yourself right now if you wanted to do any kind of basic crypto with rust, include a C library and wrap it with boiler plate just to get it working.

> which is probably what good Go and JVM languages do

Nope, Go implemented their own crypto:

http://golang.org/src/pkg/crypto/elliptic/p224.go

http://golang.org/src/pkg/crypto/

Golang had subject matter experts to do that work.

Here's the go crypto, written in go (and some hand-crafted assembly):

https://code.google.com/p/go/source/browse/#hg%2Fsrc%2Fpkg%2...

and, in case you are curious:

https://groups.google.com/forum/#!topic/golang-nuts/0za-R3wV...

There must be external libraries you can use. Or curl.

There may be further reasons they don't want to ship crypto libraries as part of the Rust language distribution when it's not yet even properly out the door. That doesn't stop third parties from writing crypto libraries in Rust and then distributing them separately from the core platform.

I can think of a couple obvious reasons why they would take this approach: (1) Crypto is hard, and they want to get the core platform out the door in a usable form on a relatively tight schedule. (2) Designating an official Rust crypto solution would be a social statement about what crypto code you're supposed to use in Rust, even if the quality of the code itself is inferior due to being newly written and unvetted. This is riskier than just using existing C libraries, because at least now they can't be blamed for any horrible security bugs in libraries they didn't write.

If you'd only read past the specific sentence you quoted, you'd see that they do have plans to build crypto solutions in Rust, but that they're taking an extremely conservative approach to making it happen, and developing it "out of tree," ie. not as part of the Rust language codebase. Moving conservatively with respect to crypto development is not inappropriate. I'm sure you understand the sensitive nature of crypto libraries and why they need to be treated with as much care as possible.

I totally agree with everything you've said and I did read the entire paragraph, but speaking as an application developer as long as crypto is an issue in Rust, because there aren't libraries and they haven't yet implemented crypto, Rust is not a good option.

I'm not trying to write a middlebrow dismissal, I'm simply explaining honestly why I don't use Rust and why I will not use Rust for the foreseeable future. The implication being that it's not just about me, but any application developer would pass on Rust as long as the only crypto options are to import c libraries. I could be mistaken in this.

If you can link against crypto libraries written in C, why is that an issue? If Rust 1.0 comes out with the SMALLEST amount of features possible that's not a problem, since you can always add more features in a backwards-compatible manner.

You can't take away a badly-designed stdlib. Now people already use your poorly-designed APIs. You have to support them for 50 years.

> If you can link against crypto libraries written in C, why is that an issue?

Rust is memory safe and C is not, so it seems to go against the benefits of Rust.

It's a pragmatic decision. Incorrectly implemented crypto has worse consequences than potential memory safety problems.

We have crypto experts on staff who could write this kind of thing, but we have a shipping schedule to meet and crypto is the kind of thing we definitely don't want to rush out the door.

Doing either (incorrectly implementing crypto or telling users to use an unsafe C library) would be highly irresponsible. I respect the hell out of you and your team but I think you are dead wrong on this one. Have your domain experts take their time and do it right.

I agree, especially interesting because in a good written crypto some functions should spend always the same time, independent of the input, and you have to know if you designed your language good enough to even make such crypto possible.

This shouldn't be a problem. If you can do constant-time operations in C, you can do them in Rust, by dint of the fact that you can more or less generate any LLVM IR you'd like in Rust, up to and including inline asm.

There are plenty of safe C crypto libraries. NSS, OpenSSL, and gnuTLS are all widely used by a LOT of sites, and, recent OpenSSL vulnerability aside, have a great track record. What would you suggest using?

lets not forget nacl/sodium :)

> We have crypto experts on staff who could write this kind of thing, but we have a shipping schedule to meet and crypto is the kind of thing we definitely don't want to rush out the door.

I hope that the "wrap C libraries" approach then is a short-term approach, and that doesn't mean that you don't plan on having those crypto experts build crypto libraries in Rust (hopefully, with compatible APIs, so that there can be a seemless transition down the road.)

That's just another way of saying that Rust is not ready for wide use. Nobody disagrees with that. The OP is basically saying that "we need these 7 things to be ready for prime time".

I don't think we need all of those seven things to be ready for use, otherwise we'd block 1.0 on them. They're all high-priority though.

This is like a checklist of all the most horrific libraries to write. I wish them luck.

With the exception of HTTP, I'd say it's more that the libraries are unsexy, not horrific.

Time is pretty horrific; I've been using the Clojure wrapping over Joda-Time, and while it's nice, this is an intrinsically extremely messy field. E.g. one that requires frequent updates for Daylight Savings Time changes.

Yeah, even at the fundamental level, there are fudge factors that vary from year to year and can only be worked out by astronomical observation.

Just measuring what is meant by 'year' is a tricky problem: https://en.wikipedia.org/wiki/Year#Sidereal.2C_tropical.2C_a...

looking at these libraries that are non-extant makes me think that Rust won't be production ready for quite a time still...

The List:

1. HTTP

2. Crypto

3. Unicode

4. I18n & Localization

5. Date/Time

6. Generic SQL

These are things you can't just throw together quickly, it takes years to do these things right.

EDIT: I see your points guys, wrapping up C code will make some of these things go pretty quickly, and your also right about a lot of production apps - at least low level systems stuff - not dependening on hardly any of these. I suppose I was thinking of this as an app developer, and if your talking about creating desktop apps in Rust, like a Browser or a SQL-Editor etc, every one of these libs has to be at least pretty darn good - otherwise its just a headache for you as a dev.

I wouldn't go so far as to say Rust will not be production ready until we have all of this. First of all, not all apps need all of these things; especially systems-level apps don't need most of this. Second, you can easily bind to C libraries that provide these things, making Rust just as production-ready as C is.

Of course, that isn't to diminish the importance of having high-quality, safe, idiomatic implementations of these things, but that shouldn't stop you from using Rust in production.

Fortunately, Rust has a lovely C FFI, so for the time being any Rust library can just be a thin mapping over an existing implementation in C (the OP even suggests this for both Unicode and crypto libs). As years go on the API can remain the same while the underlying C bits get gradually replaced with Rust.

It depends on what your production needs are, and how much of the gaps are filled in by unofficial libraries in the meantime (e.g. there are Rust crypto libraries, but none that are officially endorsed). Production-readiness comes in degrees, especially for a community project like Rust that has a relatively small paid team.

Although the internationalization story looks pretty bad, I'm heartened by my firm impression that no language does it well, that serious apps (like Firefox) tend to roll their own. One of our driving philosophies in Rust is to get actual production quality subsystems into the standard library by proving them in applications like Servo.

Out of that list, C++ has standard crypto and date/time, but it's production ready.

There's nothing stopping Rust from having a healthy ecosystem of third-party libraries that provide that functionality. Yes, including C libraries.

This is very effective because it basically creates a land-grab for any of those libraries. Most people make a prediction that a library they are thinking of writing will be needed/popular, this outright states it, creating a low risk incentive. (Of course it still has to be well written, which is hard).

Poor date/time libraries are a major source of long term pain for developers. Hopefully Rust will get this right early on!

It amazes me that this language was supposedly written to be the groundwork for an entirely new html engine and yet there has never been a really nice HTTP library; built in or third party. Look at some of the last language hype parties: Node and Go; they both didn't even hit alpha stage before they had fantastic HTTP/networking libraries built in.

I get that Rust is targeting low-level, "systems" programming, but I really don't see how you can't also be a very performant server language.

libgreen/libnative (the modern Rust I/O story) have only existed for a few months. The focus was on the language before the I/O runtime, as the entire runtime is written in the language.

  > they both didn't even hit alpha stage before they had 
  > fantastic HTTP/networking libraries

Rust is not yet alpha. It's still pre-alpha.

In fairness though, it's very unlikely that Rust will be distributing an official HTTP library by 1.0 - it's a big, difficult, library, the type which deserves to prove itself in for awhile before settling on. For these types of libs my planned trajectory is to integrate with Servo, make them installable via the package manager, let them stew in the wild until we're confident, then 'flip a switch' and declare them official.

Given that people are working on a high quality package manager, what do you gain when you "flip the switch" and make it official, rather than just having it be the most used HTTP package?

You don't gain much at that point, and that's the aim: a process that allows the best libraries to smoothly progress from community projects to official components, without any major disruptive events along the way.

SqlAlchemy core is a good system to learn from. Great, in fact. It should be fairly easy to preserve the idioms in a way that feels natural in Rust too.

You should talk to the golang core folks about time - I believe they have well-informed opinions.

Looking at their language design – probably not.

Reply on news.ycombinator.com