Rust is a hard way to make a web API
macwright.com
macwright.com
Right now, I'm writing a client for a virtual world in Rust. There's a big need for concurrency, but, unlike web stuff, it's all tightly interrelated. There's a GPU to keep busy. There's network traffic of several different kinds. There are compute-bound tasks which need to be kept out of the frame refresh loop.
Rust is good for this. The existing C++ client is too much of a mess to make concurrent; people looked at it and gave up. In Rust, it's coming along nicely.
Use the right tool for the job.
edit to respond: as the commenter below pointed out, dynamic languages can express sum types just fine (though statically verifying e.g. null checks isn't possible without some sort of gradual typing effort). This is why they are superior to traditional statically typed languages like Java. But even better are languages which have static sum types.
What I am saying is if you have static types (good) but don't do anything else, you loose sums (bad). You then need to add (back) sums.
Firstly, dynamic languages are typed. The word "untyped" has a very specific meaning in a narrow branch of computer science, as in "untyped calculus", which refers to a situation in which the types of the syntax tree nodes of the program are not being taken into consideration. (Which doesn't mean they don't exist, by the way!)
In sofware engineering, an untyped language is something utterly unsafe, like assembly language or BCPL, where every value is just a machine word, and the meaning/type of a machine word is just that of whatever operation is being applied to it at a given spot in the program.
Dynamic languages are typed, and even to the extent that some of them do assign type to the nodes of program (type inference).
Secondly, a "tag" is an implementation concept. A tag typically does not give all of the type information about an object, or not about every object kind. There may be more than one kind of tag used. For instance, a value might have a two-bit tag indicating a crude classification of a value into four categories. For an unboxed fixnum integer, that indicates the exact type (fixnum), but most heap objects might be lumped into the same category accordint to that tag, distinguished by a more comprehensive type tag that might take on, say, one of twenty values. That tag is also not complete information for all objects. For instance for an OOP object, the tag might indicate "this is an OOP object", without regard for its class: all OOP objects might share the same tag. Yet, at the language level, their type is their class. Another example is functions. All functions might have a tag indicating "this is a function". There might be a separate tag for things like compiled function, interpreted function, foreign function or whatever. But the type of a function takes into account not just that it's a function, but other properties like number of arguments. In many dynamic languages, you cannot call a two arguments function with three arguments. That check doesn't come from the tag, which doesn't have that information.
The type of an object in a dynamic setting goes beyond the tag. For some objects, the word tag tells everything, and the heap tag does for others; but not for all object kinds.
When the dynamic language implements a type inference system, the truths which that inference system works with cannot simply be reasoning about tags. The representation of type has to include concepts like the class of an object, or the number of arguments and types of a function and its return type.
Sum types are a zillion times more important.
I wrote this back in 2014 comparing (early) Rust, Go, and Haskell, but I think it's still more or less accurate. https://yager.io/programming/go.html
Anyway, this blog-post seems to rally for a better eco-system work on Rust part. Being a new language without a rich parent (like Go's "google" or C#/Typescript's "MSFT"), the situation is somewhat understandable.
Where I work we've shipped Go APIs using just the standard library running on like 1/10 of the hardware we were using in the .NET (not core) world and handling 10,000s requests per second. Perhaps that was just because we had really old shitty .NET code though.
Now 4 years of golang. It beats all of them at development pace and ease of integration, especially since modules are working as they should. The killer features of golang: modules, http stuff, channels, ssh in stdlib, fantastic, easy to use crypto.
One of our Java backends handles a few thousand requests/second with no real optimization.
The ORM story in Go is terrible compared to Java and C#. Probably due to lack of generics. I won't touch it personally till this changes
The prevalence of swagger disagrees.
> Use the right tool for the job.
Rust will be the right tool for the job when the library ecosystem evolves beyond just Actix. Give it time.
FWIW, I've written half a dozen web services in Rust [1] and it's a breeze. The type system is incredibly expressive and lets you accomplish things with clarity. I already find it an appropriate tool for backend web and service development.
[1] eg. https://vo.codes is 100% Rust
For more depth, check out FreeCodeCamp's course: https://youtube.com/watch?v=YS4e4q9oBaU.
I think you mean to use the right medium for the project. Languages are a communications medium. Your compiler, linter, editor, debugger, those are your tools.
Also, they are tools in a much more narrow sense as well, but that is left as an exercise.
1) You don't have to touch Content-length since it's computed. https://golang.org/pkg/net/http/#ResponseWriter
2) ETAGS, expire, ect ... are not managed by the std lib, as it should not.
Which looks like the right choice. ExpressJS has some helpers for it but it's a soup of unpredictability and shitty performance. You'd want something as simple as uwebsockets.js
What does Google do on servers that has not already been served by Java and C++? As far as I know, the vast majority of code both existing and newly written at Google is Java and C++, not golang.
Keep in mind, I use rust and c++, primarily. I just look over the fence at go and can appreciate what it does well.
They realized C++ sucks for simple services and due to the Oracle litigation they were trying to get away from Java where possible.
Having written backends in C++, Java, Python, Rust, and Golang myself, Golang is far and away the best tool for the job in my opinion.
However, Koltin and Android depend heavily on the JVM eco-system, and Google is certainly not going to rewrite everything in Koltin/Native or start using ART outside Android.
Even if you castrate yourself to Java 7, whatever works depends pretty much on the Android version.
not a lawyer here, but isn't the litigation related to Android Java-clone, not 'server-side' java-usage?
I think to use them when I'm trying to "show my working out" and it takes me back to algebra class days so I don't mind
I think "Go uses single letter variable names" is not so true in practice. There's a balance, in every language, between when to use short and long names. Go's balance is tilted (a good bit) more towards short, but it's not a hard rule.
There's significant mental overhead in reading `numberOfSubprotosOnProto`, as well as time hearing it in my head. I'd rather see `n` declared, used a few times in the next <10 lines, and then disappear. If there's two similar concepts in the same function, use slightly longer names, sure. Certainly not m and n.
I think that's the same reason math tends to use single letter variables. "Taylor series are easy! Just memorize: 'f_series_degree(evaluation_point, expansion_point) = sum(derivative_order=0..series_degree, (evaluation_point - expansion_point)^derivative_order * f^{derivative_order}(expansion_point) / derivative_order!)'
No thanks, I'll take f_n(x, a) = sum(i=0..n, f^n(a)(x-a)^n/n!).
I'm not saying more descriptive variable names doesn't help, but saying that you are not into mathematics makes your point moot.
CS papers are the worst in my experience. I know lines have to be kept short, but I think some people believe a tiny code block makes them look like a better programmer.
I like how Swift does it[0] — it reads nicer than any other language I've used.
This is news to me. I use JOE and, occasionally, vi (sic). But when I use short variable names it's often because they're idiomatic and expressive. In C for example, i, j, n, src, buf, cp, etc.
Many moons ago I used IDEs. Eventually I tired of installing, configuring, and maintaining bespoke environments.
I'm confident I'm not alone.
https://insights.stackoverflow.com/survey/2019#technology-_-...
I can't imagine working that way myself. I need syntax highlighting, code completion, and go to definition at the minimum. Another newer feature I like is spelling and grammar checking of code comments.
Edit: Have you given Sublime Text 3 a shot?
Let's just say that it's a big world. People programming Excel spreadsheets aren't using vi, either. I won't dispute that.
99,9% of games that you describe are built in C++, it works fine. Modern engines are doing that without much troubles.
C++ programs tend to be heavily inheritance oriented, with huge lists of includes and inheritance. Decoupling things to parallelize them is very difficult in that environment. Rust is mostly a single assignment language with move semantics, which encourages creating objects (yeah, the Rust crowd calls them structs) and passing them off to something else in a somewhat functional style. So you get programs that are less tightly coupled. That's useful if you find you want to pass an object to another thread, put it on a queue, etc. for performance reasons.
Any dangling pointers or cross-thread references will be caught at compile time. That alone makes development far easier.
So it is always a matter of equating the value of a full rewrite and how well the checkers are able to provide valuable feedback.
Which I concede Rust will be much better, however then there is the whole loss of ecosystem to also consider.
> Which I concede Rust will be much better,
Aren't these two statements contradicting each other?
Games also really don't care since they are single use applications. You are not going to run something else cpu intensive in the background while playing.
Do you agree with the article's recommendation of Rust for command-line tools, or do you think Go is the right tool for that job as well?
Or perhaps I should stick with C++? For one thing, it's nice to be able to re-use code between command-line tools and other C++ programs, which AFAICT would be a bit harder between Rust & C++ and much harder between Go & C++.
I think between http, encoding/json, and the io libraries, it’s also very simple and straightforward to do most of the things a typical command-line program does, in a cleaner way than shell or Python (both have their place) and with less fuss than Rust or C++.
Lastly, does anyone have any resources for actually writing concurrent Go code? I have been spoiled by so many of the underlying libraries “already” doing this for me.
Certainly, it's still ways better than C. But the resulting boilerplate and copy-paste programming ought to be a source of errors.
(I'm happy to see the generics proposal coming closer and closer to a future feature.)
PUSH EBP
MOV EBP, ESP
SUB ESP, 12
Which just demonstrates why slogans are not a good substitute for thought.
They're good for programming a community though.
Definitely not, unless you are google.
Go is great for hardcore backend and systems programming, as a replacement for c/c++ or Java.
For most companies over there (which are not Facebook, Amazon, Google, etc) what you need is just f**ng python/django, ruby/rails, php/symfony, etc.
P.S. You probably don't need kubernetes and monorepos either.
A typical Golang CI/CD pipeline should - compile without CGO enabled and with go build -tags netgo -a -v so that the binary can run on in a minimal docker image like alpine or scratch - should be compiled and tested using the race detector - should be linted using golanglint-ci
All of these flags and linters considerably increase the build time for deployment to production
The rust compiler effectively gives me the same outcome using the type system and the compiler advancements when targeting release musl.
I have seen many Golang projects where the race detector was skipped because the code base was to large or it was flaky.
One the other hand I do miss the code coverage tools. Code coverage for rust seems to only work on Linux
Developer productivity is unmatched with Go. Focus is almost always on the problem to solve not on language features (I'm happy for generics to come but really don't need them), memory management / borrow checker or build tools / config (gradle et all).
Rust is a formular 1 car. Use it when you need it and be aware that it has substantial costs developing and maintaining (immature ecosystem).
There are a bunch of things rust has which makes it attractive over go for these types of apps, from a practical perspective it tends to be better for a team to use a minimal number of languages across their stack.
The lack of generics means that each numeric type has it's own signature, if you deal with a lot of numbers with different precisions - you'll have a bunch of identical methods lying around just to support alternate precisions.
The lack of function/closure serialization in Go means that "moving compute to data" ala Spark or Hadoop becomes exceptionally difficult.
As native code, rust interfaces cleanly with C and can run in a few more places than Go - there are some active efforts to even support GPU kernels.
Go can't really do any of the three things above today, and the community is averse to include new features into the language for both good and bad reasons. Java can do the first two, but struggles in the last category.
Rust has an interesting opportunity to be a modern C which does the things java does and the things C/C++ do. But the library support and learning curves would need to improve dramatically for it to get there.
There are projects and examples to do all three but we don't want to assume one solution is right for all domains.
We take this to the extreme...Juniper doesn't even require a web server and isn't tied to a particular serialization format! Of course, we provide optional integrations with popular web frameworks using json but the key is they are optional.
Just want to call out that I've heard a horror story of a very bad security bug due to implicit shared mutability in Python.
(vague to protect the innocent)
> Heck, if you ask some people, Rust is less secure than a GC’ed language for web apps if you use any crates that have unsafe code - which includes Actix, the most popular web framework, because unsafe code allows things like deferencing raw pointers.
You can write unsafe code in Python (and some of the web frameworks use this) and nothing helps you with auditing it, unlike unsafe blocks in Rust.
> You can write unsafe code in Python (and some of the web frameworks use this) and nothing helps you with auditing it, unlike unsafe blocks in Rust.
This was the dumbest part of TFA Python and Node both have parts of their runtimes, standard libraries, and extension libraries written in C and C++. Last time I checked, C and C++ allow dereferencing raw pointers.
So, Ruby has Rails, PHP has Drupal/Wordpress/etc, Python Django. However, neither Go nor Rust have a mature framework that makes it easy to spin up a CRUD app. I'm hopeful that some sort of project in Rust will rise up along the lines of Flaskrestplus in which resources are the primary abstractions. Resources could have codegen'd frontends built using something like react-admin so that you could build a fully functional web form in 100 loc or less. There's no reason the frontend bits couldn't be shared by Rust and Golang (and C++, D, Haskell, etc) frameworks.
While I use neither Rust nor Ruby, they seem to have a very different attitude towards explicit vs. implicit expressiveness and exact, correct code. I'd trust explicit code much more than spooky-action-at-a-distance and it-works-dont-ask-me-how.
I'm still choosing to write some web stuff in Rust for the experience, for the fact I can compile parts to .wasm and re-use them in a browser too, I enjoy tuning things for performance, and I am not under time constraints for it.
I took a quick look at their auth docs, and while the docs seem pretty detailed, I noticed it involves multiple packages (@nestjs/passport, passport, passport-local, and their corresponding types) that you then have to glue together into a full solution. The instructions also apparently only show you how to store passwords in plain text, and using something like bcrypt to do it yourself is an exercise left to the reader. Not rocket science, as this [1] (older) article pointed out, it's hard finding quality auth-related tutorials in the Node ecosystem. With Django a lot of this stuff is built-in and solidly implemented. Granted, this was after a quick look at the docs, so my impression might be off.
[0] https://docs.nestjs.com/security/authentication
[1] https://hackernoon.com/your-node-js-authentication-tutorial-...
I love typescript, but the ecosystem is without a doubt one of the biggest detractors.
With regards to TS, generally, it takes a lot of effort to get it to work. More often than not, getting to that point in a way that is actually useful/practical and cost effective, requires yet another library to be installed.
Just look at the TSLint/ESLint migration problems, the popularity of all those TS runners such as ts-node/ts-node-dev/tsx-watch/ts-babel/etc., the complete lack of interoperability between all those separate pieces.
Many issues and even proposed fixes to parts of the TS ecosystem - which is all community ran - are either ignored, left open because no one seems to be able to figure it out, or deemed to be out of scope.
I don't know how to fix this honestly. Some leadership from Microsoft would be a good start.
All the problems I've had were with JS and npm. TS has been amazing and worked flawlessly. I just can't understand your experience at all.
[0] https://github.com/blitz-js/blitz [1] https://github.com/redwoodjs/redwood
/* * @type {Map<String, Number>} */
const map = new Map();
Like that.
Express JS is still shit, koajs and fastify are a little better, I used to use nodejs's internal http and https libraries directly but these days uwebsockets.js is what you'd want since it's mature enough already (used by top crypto exchanges).
https://github.com/uNetworking/uWebSockets.js/
And oh it comes with TS types which are cool.
The last time I tried to use the JS/TS dev tools, I wasted many hours because the tools did zero validation of config files. All of the tools happily silently ignore misspelled config key names and invalid values. This makes simply debugging the tools a huge time sink.
Yes, they might not match Rust down to the microsecond or byte count, it doesn't matter in distributed computing.
There are bigger fishes to fry as distributed computing problems.
And Rust type system for preventing data races among threads doesn't help when what is being used is distributed IPC across a cluster nodes.
At one point I did a comparison between a mono/Nancy web service and the same web service, feature for feature, built in NodeJS, Rust/rocket, and Go. I built a few different endpoints that did different types of work and load tested them, profiled memory usage, etc. I know mono and perhaps Nancy weren't the best point of comparison, and it's been some time since, but Rust/rocket really impressed me then, including on axes like ease of development and code readability. The perf deltas I saw could matter quite a bit at scale, allowing for fewer/cheaper servers.
I've enjoyed using Rust for a few things since, personally, and am so far pretty unbothered by its shortcomings.
I really do not understand how convoluted javascript frameworks that require a thousand tools and checkers and dependencies just to put up a hello world are sold as "easier choices", specially when there is no mention of SPAs anywhere.
Is it that hard to deliver static or dynamic HTML+CSS?
I highly encourage everyone to watch Steve Klabnik's video on how Rust Views Tradeoffs [1]
Rust has these core values that they refuse to compromise on in this order of precedence:
- Memory Safety
- Speed (compiled binary execution speed, not compile time)
- Productivity
And because of these core values, other aspects emerge such as the long compile time as they value memory safety as the language's highest core value and are accepting the tradeoff of long compile times.
The talk goes on to say, we should ourselves reflect and ask what our core values are that we can't compromise on? What are the secondary values and so on... And then find the language/tool that lines up with our values.
I really can't emphasize enough to go watch the talk.
For adjunct pieces such as web app authentication, including some common features such as user sign up via email or phone, recover a password, multi-factor auth, social sign in, and the like, there's not yet a good solution (AFAIK).
I will gladly donate money toward developers working on this, especially if it's something simple such as Elixir Phoenix `mix phx.gen.auth` doing a one-time setup with good defaults, then fully customizable.
But if someone wants to take up the project I don't mind donating the code.
If anyone ever saw my version years ago[1], it's effectively Jelly 2.0.
I re-purpose abandoned code all the time. It's not a moral failing to release unfinished code, especially if you label it as such.
I’ll consider dropping it later tonight or this weekend.
Are you looking for a framework-agnostic solution here, or one Really Good Solution for the leading framework (whichever that may be)?
It's used by at least Actix-web, Tide, Hyper, and Warp. Generally, this crate's types aree re-exported from the main framework so you can avoid adding extra dependencies to your application, but AFAIK the compiler can "see through" the type alias to see that the types are the same.
I think the older generations of web frameworks (like Iron, Nickel, and Gotham) might not have leveraaged the http crate, so your point still stands.
It not help rust still lack the "#1" in some key areas but maybe what we need is to people to voice the interest here.
I'm in the fence about doing something that could be the "auto-admin" alike django, and the maintenance cost is what make me doubt this. In the other hand, maybe following the model of tailwindcss with a full open source + a very nice styling to get the proper founding will help (and btw, i plan to use tailwind)
One problem that I noticed, which may become a serious issue in the future, is that a lot of libraries add abstractions via macros. Macros don't always compose or debug well. I had some experiences getting errors in my macros that had extremely confusing debug messages, often pointing to files that didn't exist. Hopefully there will be a trend of moving back to plain old functions. Nom for instance has converted its combinators into regular functions, which I like a lot.
I'd love love love for there to be a language that gets in between Rust and TypeScript. It'd have the strictness over mutability like Rust, but the GC of JS/TS. It wouldn't have 3+ string types. It'd have enum variants and pattern matching but maybe not macros. Gleam is pretty close!
[0] https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in...
I am really really keen on developing something like this, along with an optional linear type system. I think this would be a cool project for my thesis but also I have a lot on my plate already.
Here's Gleam for anyone interested: https://github.com/gleam-lang/gleam
I don’t mean to criticize too harshly; it’s a great ecosystem that I really want to use. It’s just not at the point where using it seems like the pragmatic, effective decision
That ease of use has a cost.
I think Rust may be a trap.
The 'feeling of satisfaction' of making it work according to those strict rules, I think might be a kind of intellectual heroin for developers.
I believe we may be vastly overestimating the value of that supposed 'correctness' - and also don't contemplate that it's only a specific kind of correctness.
Developers are the kind of people that will naturally hone in on solving problems and building things, and our intuition may not be remotely attuned to necessarily creating value. We'll build some 'perfect thing' long before we build some 'useful thing'.
I've become very skeptical of rust for this reason, it's so addictive to a certain personality type that I wonder if there is true reasoning behind the need for hyper strictness.
I don't think we need Rust actually, I think we just needed a 'clean' version of C++ with some nicer things around pointers and that would have been fine.
Edit: Should say I'm not against Rust, it's super cool, and likely has applications. I'm just skeptical of our reaction to it. I think it's 'borrow checker v1.0' and future languages may improve on that.
The C++ community has tried this approach, and the outcome (viz. the C++ Core Guidelines) while definitely useful was far less "clean" than Rust itself. You're definitely right that some developers overestimate the need for complex solutions, but this failure mode seems rather more common in the C++ community and it's a bit weird to criticize Rust for it.
It's not about that. Whole your comment is based on this false statement .
But I also agree with the other posters that this is a bit missing the point about what type and memory safety are supposed to achieve.
In all seriousness, if type safety feels like a burden and not an aid, then either you're using the wrong language for the application or the wrong language for the developer.
Don't forget that Rust exists to solve a problem. And that problem for the most part is not web development.
You can write a small script or tool in any language, probably even assembly if you have the experience.
Bigger applications, developed by multiple people, and longer-term-maintenance are where it gets interesting.
Then, when you suddenly have to refactor and migrate ten thousands of lines of code, the difference that "strict rules" make becomes very visible.
How Rust will do in this regards only time will tell.
It needs another lifecycle of the language ("Rust 2") and the same for the wide used libraries before it can be compared to languages like C++, Java in a more meaningful way.
The presence of _unsafe_ is okay, it just means that this is a part of code, that the compiler cannot verify itself. Usually the unsafe part (verified by human) is wrapped in a API that is safe to use.
PS: You can still have memory leaks in GC-d languages by reference cycles.
I don’t think so. It explodes the attack surface. In safe languages like C# or Java there’s no unsafe anywhere, not even in standard libraries. These runtimes are safe all the way down. The only attack surface is the VM itself, but these are very small only a handful of instructions, and are tested and audited really well.
> You can still have memory leaks in GC-d languages by reference cycles.
No, you can’t. All modern garbage collectors collect reference cycles just fine. They don’t just count references; they actually traverse these graphs.
An optional feature. Very useful for embedded and similar, but for general-purpose stuff like the web you never need anything unsafe.
> it's pretty common in both of these languages for there to be C libraries involved somewhere in the stack.
Negative. The complete stack is open source. You can browse the source code of the standard library at https://source.dot.net/ You only gonna find unsafe/dllimport for IO parts of that library where they integrate with file systems and such. All their core components are 100% managed code.
Then there's crypto, for which the corelib defers to the OS (CNG / openssl) because doing crypto in managed code is hard.
But forget all that; even pure C# code isn't safe because it occasionally needs to hook into the CLR for unsafe things. Eg https://source.dot.net/#System.Private.CoreLib/Dictionary.cs... - unsafe code called from Dictionary.Remove
> unsafe code called from Dictionary.Remove
For creating, and checking for, empty reference.
Rust standard library uses unsafe way more. That design has consequences, e.g. check this https://medium.com/@shnatsel/how-rusts-standard-library-was-...
It's somewhat harder to screw up when you can rely on VM guarantees, and don't need that level of trust in the libraries you consume.
>but for general-purpose stuff like the web you never need anything unsafe.
... and:
>> it's pretty common in both of these languages for there to be C libraries involved somewhere in the stack.
>Negative. [...] All their core components are 100% managed code.
... so I posted a correction.
A garbage collector won't help you avoid a very common memory leak of having an unbounded cache.
It has, but you don‘t need it to implement data structures. Standard library developers don’t need it either, they all implemented in safe code.
Java will have a safe foreign memory API soonish.
You can configure the compiler, runtimes or web servers to refuse to load tainted unsafe binaries.
EDIT: I didn't list Java calling C/C++ because I think? it's not very common in Java libraries (through it's in JVM).
C# has a lot about that, but whatever code you going to generate is CIL code. Runs within the same VM with strong safety guarantees.
> but one which is similar bad
Not anywhere close. Rust is unsafe all over these crates, both standard library and third-party ones. That’s not just theoretical stuff, it has quite a history, see e.g. https://medium.com/@shnatsel/how-rusts-standard-library-was-...
You still generate code at runtime and there had been multiple Java vulnerabilities where features like "runtime code loading/generation" and "reflections" lead to RCE vulnerabilities.
And sure you can RCE bytecode but so what, it's still code which can do anything your application can do. I.e. the same as you have with "unsafe" attack vectors. And while you could try to use the VM as a security sandbox it requires additional work and at lest for Java is known to not work well and if you do so you can also spend additional work to sandbox binaries...
And then C#/Java libraries still do bind to C/C++ code, and their VM is still implemented in C/C++ and neither VMs are meant to be a sandbox to allow running untrusted code (both had some features for this, in both cases but especially Java it didn't work out well and by now both have dropped/deprecated it).
> https://medium.com/@shnatsel/how-rusts-standard-library-was-...
Sure, there had been a single bad security vulnerability in the standard library on stable. But so what. There had been more then one or two in fundamental parts of the JVM and probably for C#, too (IDK).
In the end neither rust's unsafe nor Java's/C#'s VMs/GC are meant as security protection mechanisms. They are tools to make it easier to write correct code. And more correct code also means less security vulnerabilities.
If you rely on any of them for security you already have lost.
Which doesn't mean you can't design VM's for safety purposes, e.g. most JavaScript Browser VM's are designed to safely isolate JS and even then there where cases of VM escapes. In even then you might want to add at least one additional layer of protection and generally run all from the outside reachable services on properly locked down systems.
I agree with that. Still, being able to implement data structures without relying on unsafe features helps a lot with the correctness.
It’s possible to write code in safe Rust, but it’s hard to implement data structures in it. At least not if one wants performance. That rust’s borrow checker is simply too limiting for them. All basic data structures like linked lists, trees, graphs, LRU caches, use tons of unsafe internally.
That’s not the case with Java or C#. These garbage-collected VMs are memory safe all the way down. It’s usually possible to implement arbitrarily complex data structures entirely in safe managed code with minimal performance penalty.
Even their standard libraries are made that way now. AFAIK that wasn’t always the case, older versions of .NET framework did more in C++ for performance reasons, but they improved performance of JIT over time, and gradually reworked the standard library to almost 100% managed code, probably for portability reasons. Vast majority of third-party libraries in their respective ecosystems don’t use unsafe either, e.g. many asp.net web hostings are configured to forbid any unsafe libraries from being loaded.
My (limited) understanding of the unsafe keyword in rust is that it’s just indicating that the compiler cannot guarantee that the block is safe, not that it’s necessarily dangerous. By this standard, every single line of any C program is unsafe. Depending on the guarantees of other languages, this is true to varying degrees. I’m not familiar enough with Java and C# to comment on them specifically, but I’m not sure the try to provide the same guarantees that rust does.
I think all python, ruby, is, etc. would be considered unsafe. The unsafe keyword in rust seems more like the equivalent of saying “my static analyzer couldn’t guarantee the safety of this bit; it may or may not be totally fine”.
That said, I do agree that unsafe blocks can be red flags. At least they can help guide you in where to start looking for potential issues.
Now, I am not sure if memory bugs are "worse for security" than type bugs or null reference bugs (in the wild or in theory). Certainly many of the more notorious major exploits in recent years come down to memory errors. But of course, the major infrastructural software affected in bugs like Heartbleed wouldn't be written in a dynamic interpreted language anyway. More generally, a language being safe against C-style memory bugs does not mean a (reasonably well done) implementation is actually safer than a (reasonably well done) implementation in C. That statement might be true for C#/Java/Haskell, but not for JavaScript/bash/Python/etc.
It just seemed odd to not mention typing discipline at all. Presumably the author has experience with frustrating type errors in Python or "object of type None does not have method xxx" - these can be really bad for security in a Flask app unless you have careful exception handling ! And, unlike Rust, Python offers very little to actually help you with the exception handling.
public class T {
Object ref;
@Override
protected void finalize() {
System.out.println("finalize");
}
static void leak() {
T a = new T(), b = new T();
a.ref = b;
b.ref = a;
}
public static void main(String[] as) throws Throwable {
leak();
System.gc();
Thread.sleep(10000);
}
}It is OK in the same way C/C++ code is OK. IMO there is absolutely no point in writing Rust if you are going to use unsafe code.
However for getting c++ish performance for a relatively high level syntax I think this fills a different niche than GC'd alternatives. The web-server libraries are still evolving, and I too would love to see a few crates designed for handling auth.
A lot of the issues with difficulty writing/safe practice will just improve once there's more examples/books/tutorials. It takes a little for this stuff to catch up.
Finally I really struggle to see the point about dataloaders - they may be a little confronting initially but its the same thing you often have to do in Nodejs. There's public examples on Github of how to use them with both Juniper and Async-Graphql.
I can understand a lot of this as "its just not there yet". Rust has a much smaller community than Golang/Python/Js, I think its understandable if its taking a little longer to have equivalents in tooling etc. I still enjoy writing it
My day job consists of a hybrid of Rust/React/JS/CSS/HTML/Ruby/Rails/SQL (in no particular order). I don't like having to remember so much. I would love to see one language be the best at everything, but I doubt that will ever happen.
I've been doing mostly front-end work for the past decade and I could use some variety.
Oddly enough, this was the exact missing off-the-shelf piece -- traditional built-in Web authn&autho, not OAuth2/OpenID/etc. -- that last week made me (reluctantly) abandon plans to use Rust for a rapid Web backend project, and at least delay trying to do most of our new software work in Rust.
The Web authn was sorta the last straw, on top of enough additional known-extra-work and known-uncertainties, so I couldn't justify going Rust at this point, for our needs right now. I decided I had to go with a framework and language that was already heavily proven (if boring) for this kind of application (and, incidentally, one of the framework's off-the-shelf authn&autho packages turned out to be a breeze to use, so far).
(Were I doing a brand new startup demo/MVP, or an R&D project within a comfortably established company, I might've taken on additional risk here, and tried to power through the extra work and unknowns, but we are an early startup nevertheless in B2B critical production already.)
(One reason not using Rust yet was disappointing to me is, just as I'd personally prefer to be mastering Rust right now, using Rust would be a recruiting carrot, somewhat like beloved fringe language Lisp/Scheme/Racket/OCaml/Haskell/etc. would be. I could also see Rust soon letting us do all our backend, much of Web frontend, embedded/appliance systems with device interfacing and some creative networking, and even mobile apps, rather than the mix of too many languages we have now. And I also suspected that most developers we'd hire would end up creating fewer defects in Rust code, since language semantics and static checking would force us to think harder before the alternative led to runtime failures, and/or to encumbering our ability to evolve things fast without breaking production.)
(I'd probably enjoy the job of helping build out the platform/ecosystem of Rust or another newish interesting language I liked. I've done building-out before, including lots of authn from scratch. And of course sometimes it makes sense, in the context of a project in one platform, to build generic pieces that would be off-the-shelf in some other platforms; but other times that makes less sense.)
Writing efficient SQL for GraphQL resolvers is more or less the same as writing efficient GraphQL for any application or API. If you just write all of your queries in the most naive way possible, you're going to have inefficient queries. If you pay attention to querying in an efficient way, you will be able to do it without too much difficulty.
Regarding safety, rust's lifetime system protects you against more than just preventing the invalid memory accesses, which GC languages also do. But several classes of race conditions that are easy to introduce in almost any other multi-threaded language are impossible in rust's type system (unless there is incorrect unsafe code). And in a web app with high levels of concurrency, that is kind of important.
That thing just works all the time. I don’t have to track down issues very often.
It was harder to build for sure, but man it is a joy to work on.
Not technically true. Rust must do a runtime bounds check when it cannot prove that the input index goes above bounds at compile time
I personally agree that the practical potential of Rust is oversold, especially in areas like kernels, low-level interfaces, etc. Bugs in those types of applications are often precisely at language and environmental interface boundaries, places where stronger type systems don't help. Similarly, AFAIU the current Rust toolchain doesn't do a stellar job at eliding as many bounds checks as theoretically possible, and I'm probably less optimistic than most about the pace of improvements on that front.
It's just like with C++ or Java. It's easy to write a small microbenchmark or program where a compiler generates code that readily outclasses a typical, textbook C implementation. But that effect never seems to scale. People end up structuring their C++ and Rust programs the same way they'd structure a Python program, and even the best compilers can't turn lead into gold.
In the django ecosystem, I can take an existing component with a vast API surface, inherit from it, tweak a tiny part of it and then put it back in the system and I'm off to the races. With rust, you have to re-implement the entire API surface to experiment with a tweak or go upstream and modify the original source, which isn't a great long-term solution -- maintaining patches. The diesel ORM is great, but it seems that each api returns a different type and doing a builder pattern generically across different functions and getting the types correct is like being in the worst C++ templating hell.
I love the speed of execution, the type checking, the borrow checker, but the coding friction is real. I'd love to see more complex examples in all the rust web frameworks that do real work with good function partitioning in addition to the simple examples.
Diesel provides the `into_boxed` method [0] for composing queries.
[0]: https://docs.diesel.rs/diesel/query_dsl/trait.QueryDsl.html#...
> Once your code is compiled, everything’s amazing! But in my case, this basic API - which wasn’t even feature-complete and was by no means a complex system - took more than ten minutes to compile...Caching helps as long as you don’t have to rebuild cached dependencies.
The author glossed over that last part, but at least from a workflow perspective, the build cache makes a huge difference. In my experiments developing a web server in Rust, I used cargo-watch (https://github.com/passcod/cargo-watch) to automatically rebuild each time I made a change. The turnaround time was usually 1-2 seconds - nearly as fast as restarting a Node server, and about the same amount of time it takes me to alt-tab and test the change. I was using Serde, a high-level HTTP framework, and several other crates. It didn't matter, because they never had to be rebuilt. My own code was small, but still.
> Rust makes you think about dimensions of your code that matter tremendously for systems programming...It makes you think about real but unlikely corner cases and make sure that they’re handled...These are all valid concerns. But for most web applications, they’re not the most important concerns.
I disagree strongly (at least about corner cases). Maybe you don't want to bother with this stuff when you're still in the prototyping phase, but once a service is fairly well established, it's definitely beneficial to be forced to think about corner-cases (both in libraries/IO, and in your own business logic that you've hopefully modeled with Rust's powerful type system). Not only is your API usually the authoritative source of truth for your application/service, it's exposed to the whole internet by design, just begging to be prodded for logic holes. This IMO is one of Rust's main benefits; it's been called "the practical Haskell" before, and while its ecosystem isn't yet the most practical one for web servers, it is still much more so than Haskell's.
> Lots of missing pieces
This is the strongest point, in my opinion. Rust's web ecosystem is definitely still in the early days, and this is partly because Rust's benefits aren't nearly as extreme in this usecase as they are in other usecases. There is for sure a chicken-and-egg problem as not enough companies are using Rust for web servers, which means not as much time is getting invested in the relevant libraries. I hope this changes; I don't know for sure that it will. It feels like it is, but very slowly. That said:
> Unfortunately, a lot of the incredibly exciting work in the Rust ecosystem has nothing to do with web application servers. There are some promising web frameworks - even a somewhat higher-level framework - but they’re undoubtedly in a niche. Even Actix, the main web framework, has a very top-heavy set of contributors.
The author failed to mention or wasn't aware of Rocket (https://rocket.rs/), an up-and-coming Rust web framework that's extremely exciting and provides a programming experience strikingly similar to that of Flask or Express. It's still in 0.X releases, the current release doesn't build on stable rustc (though the master branch does!), you're still going to have a hard time finding SDKs for auth and payment and cloud, etc.
But the important thing is that it shows what's possible. Web servers can be ergonomic to write in Rust, benefiting from its wonderful type system and performance, with very few sacrifices. We just need the ecosystem to catch up. Hopefully enough companies will realize the opportunity and that will happen.
In summary: Yeah. Rust is a hard way to make a web API right now. But I don't think it has to be.
Just use stuff that works, and that most people already know. Python, Perl, Ruby, jQuery, PHP, etc. Any of those have all the tools you need, proper documentation, millions of Stack Overflow discussions, and exceptional ecosystems and communities.
I am a Perl enthusiast since more than ten years. Personnal opinion, feel free to ignore it. It got the incredible [Mojolicious](https://mojolicious.org) framework that makes possible to deploy a full API through a oneliner. I know everybody hates Perl (especially for web development), but I am really tired of all the re-inventing the wheels.
I would never study another languages. I am forty. I am tired.
Most of the stuff you mentioned is also highly inefficient and requires much more computational resources to reach the same level of service.
Java is also conspicuously missing from that list, even though it has a solid infrastructure and ecosystem.
Who spoke of efficiency. Everything is inefficient. Are cars efficient? No. You use less time at walking your way than using a car and working to pay your car. Plus they are dirty.
I have to admit that I forgot Java that have the most efficient object system in the world: btw I got trained on it and spent half of my career coding Java. I just tried to quote stuff I wasn't familiar with.
And even the real time messaging stuff? It probably depends on how real time you need, e.g WhatsApp is heavy on Elixir I believe and is the better choice for real time messaging due to its amazing actor concurrency model.
Rust is a niche language for most programming domains imo but when you have a use case it really shines. But it also is such an attractive language because it brings out the engineer in all of us.
The addicting process of optimizing every bit of code when in reality you probably should of just spun up a Rails app for your service and scaled horizontally eating your sadness of the performance engineer within yourself over the business logic that needs to get done yesterday.
https://elixirforum.com/t/facebook-is-writing-a-new-statical...
This is what has me contemplating moving away from Elixir. The top three web back-end languages, judging by the availability of certain SDKs, seem to be Python, Java, and JS (via Node). Now I just have to choose between those three, for a full-stack, mostly server-rendered, web app.
TBH I think this is the primary reason why Rust isn't as nice as e.g. Node. Convenient (and safe and decently performant) web libraries need to do a lot of complicated stuff under the hood, and the community hasn't (yet?) put in enough effort to reach parity with more web-oriented communities.
(edit: I'm honestly kinda confused why this is getting downvoted so quickly. do people disagree that Rust can be nice?)
> the community hasn't (yet?) put in enough effort to reach parity with more web-oriented communities
Rust is a relatively new language, and these things take time. The community is amazing and is putting in a lot of effort, but you have to be patient.
It is approximately the same age as Go and Node. Unless there is something fundamental to Rust that prevents the creation of what the parent describes as a convenient web library, then not putting in enough effort yet to this point is the only reasonable explanation. And a good explanation at that, as it is understandable why convenient web libraries has not been Rust's focus. There are plenty of good tools for that job already. The Rust project seems to be more concerned about creating good tools in areas where good tools are currently lacking.
If downvoting was supposed to send some kind of message, I'm not sure that would explain it. Of course, all downvoting tells is that someone accidentally hit a button when they were trying to scroll through the page, so this is all moot anyway.
While the Rust project is from that timeframe, the language internals were rewritten almost from scratch shortly before the 1.0 release, which dates from 2015 and is a more informative comparison. Nothing comparable occurred wrt. Go or Node.
That only speaks to priorities, though, which is exactly what was asserted earlier. If Rust's goal was to provide the best convenient web library, rewriting language internals would have been put on the back burner in order to focus on building the web library. However, that is not the Rust project's goal, so it was reasonable for people to choose to focus on something else instead.
> Nothing comparable occurred wrt. Go or Node.
Go (gc) was rewritten in Go. Completed in, funnily enough, 2015. I'm not convinced that year has any significance. The inception of the project is where the direction of the project is largely established. All of these tools were created to solve a particular set of problems and their course reflects that.
For anything else, just use a language with automatic memory management and tooling capable of doing AOT/JIT compilation.
Doing it in Rust while doable, feels more like trying to prove a point than doing any kind of usable work at scale.
I really hate that I needed to do this, but I recently acknowledged that my 2016 macbook pro just isn't fast enough for rust development to feel good. I replaced it with a ryzen 5800 based desktop workstation for dev work. With my new machine a full build of a rust project I've been working on dropped from 2 minutes down to 20 seconds. Its an absolute delight and I highly recommend doing the same if you're writing rust regularly and can afford it. Hopefully the next few years of CPU improvements will forgive our collective inability to engineer CPU-efficient compilers.
Exhibit A, in our recent history of compilers we've had lots of examples of compilers / runtimes which seemed like they were pretty close to the threshold, only for another compiler to come out of nowhere and compile many times faster. Examples: (everything before V8) -> V8, Lua -> LuaJIT, Gcc -> clang (when it was new). And this is cheating but I'm doing it - every C compiler -> Go's compiler.
So the question is, how surprised would you be if some serious, experienced compiler people were capable of making a rust compiler that was 3x faster than rustc? Personally I would be not surprised at all. I suspect a lot of rustc's slowness comes from a combination of unavoidable time spent in llvm plus all the work rustc needs to do to convert rust to llvm IR. Its a bit of a smoking gun that most compilers built on top of llvm are pretty slow - swift, c++, rustc, pony, etc. And all the fast compilers have their own backend (go, v8, luajit, jai, etc). So yeah, I'm hopeful for MIRI and cheering from the sidelines.
Along these lines, Rust developers are working on adopting cranelift (a compiler framework used for JIT in Firefox) for quick debug builds, where runtime performance is not critical.