Hacker News Reader

Comment by kaladin_1 | Hacker News Reader

I have no problem with people rewriting their projects in whatever language they see fit.

What stood out for me in the article is him saying that it's difficult getting developers with experience in both Python and C++.

So, I wonder, if his in-house devs could pick up Rust that they previously couldn't write, why does he think he can not hire a good programmer and charge him to learn the stack the company uses. Why must they employ someone that already writes Python or C++.

Is Rust such a straight-forward language that people new to the language can write a very performant programme

View on HN

Despite Rust's steep learning curve, it's also paradoxically easy to add novice Rust programmers to a project.

This is because inexperienced Rust programmers are relatively harmless. Noob mistakes won't compile, rather than running into dangerous gotchas. You can tell noobs not to use `unsafe` (and there are ways to enforce that), and mostly they'll just write inefficient or non-idiomatic code, but the code will be free from data races and memory corruption.

The strictness of the Rust compiler is quite the opposite of something like the C++ Core Guidelines where the majority of the rules aren't enforced by the compiler, and have to be in the programmers' head first.

Noobs make lifetime errors and fight the Rust compiler, but imagine working with a compiler that doesn't tell you when you have lifetime errors.

I think it's great that Rust is safe in that way, where you can have novice programmers write code and you can be pretty sure it won't break anything else.

I recently spent some time fixing performance issues in some novice Rust code, and while the code was pretty clearly written by someone new to systems programming it still all worked fine - https://jackson.dev/post/rust-coreutils-dd/

Memory sanitizers, address sanitizers, leak sanitizers, threading sanitizers, undefined behaviour sanitizers. The visual studio core guidelines checker. The clang-tidy core guideline checker. I could go on but my point is, the landscape does not really look like how you've painted it.

I know about these, but there is a marked difference between Rust and these tools.

Static analysis tools have much harder job analyzing C++ (aliasing and escape analysis are way harder, and static analysis of thread-safety is basically impossible due to lack of thread-safety info in the type system). The results are a trade-off between being sparse or having false positives.

The sanitizers only catch issues they can observe at run time, and that relies on having sufficient test and fuzz coverage. Some data races are incredibly hard to reproduce, and might depend on a timing difference that won't happen in your test harness.

OTOH Rust proves absence of these issues by construction, at compile time.

It's like a difference between dynamically-typed and statically-typed languages. Sure, you can fuzz type errors out of JS or Python, but in statically-typed languages such errors are eliminated entirely at compile time. Rust extends this experience to more classes of errors.

The results are a trade-off between sparse or having false positives.

Rust just takes the other side of the trade-off, and will reject valid programs. Hence why the unsafe keyword exists, and why tools like Miri (https://github.com/rust-lang/miri) exist specifically for rust.

Sure, that's the consequence of Rice's theorem. "Is this program correct?" is formally Undecidable, to dodge that we split the programs three ways: Correct, Not Correct, shrug emoji.

It's obvious that Correct programs compile, Not Correct programs result in a compiler error, but what do we do with shrug emoji ? Rust says those go in the "Not Correct" pile and get a compiler error. C++ says - really, I'm not making this up - that they go in the Correct pile.

There's an immediate short term consequence, but also, after decades, a long term consequence that's arguably worse. Short term, C++ programmers can't know if their non-trivial program is Correct. It might be complete horse shit, the compiler won't necessarily tell them.

Long term, C++ grows worse and worse because there is no reason to shrink that shrug emoji category. All the programs in that category act as though they're fine, so there's no pressure whatsoever to improve the language standard, the compilers, etc.

> Rust just takes the other side of the trade-off, and will reject valid programs

This has always been such a weird claim to me because it's not clear to me what's meant by "valid". My instinct is that it would be hard to define valid in a way without making most languages either accept some invalid programs or reject some valid ones (the exception being defining "valid" as "anything that language X accepts", but that wouldn't really say anything interesting about Rust). Sure, you could draw the line so that Rust is the only one that rejects valid programs, but is it worse to be the only language that rejects some valid programs than to be a language that accepts invalid ones? The alternative is that all languages accept some valid programs, at which point there's nothing specific about Rust that's worse until you start specifying which valid programs each language rejects. If there's a way to define "valid" so that Rust is the only one that rejects valid programs but other languages accept all valid and reject all invalid programs, I think it's non-obvious enough that it should be explicitly stated.

That definition of what is valid or not depends on the type of checker. When you're talking about Rust's borrow checker, then you can take any property that it checks for, such as "there are no aliasing mutable references" or "any variable that is read from has been initialized" and build an example for it that would be correct, but is rejected by the borrow checker.

Here is an example where a variable is definitely initialized, but it will be rejected.

  fn main() {
      let foo;
      if true {
          foo = 1;
      }
    
      println!("{}", foo);
  }

> Rust just takes the other side of the trade-off, and will reject valid programs.

Are we still talking about ease of add novice Rust programmers to a project?

Over the decades, C started to reject more and more "valid" programs, and you have to use explicit casts to go from one pointer type to the other, or from an integer to a pointer. For obviously good reason.

> and that relies on having sufficient test and fuzz coverage

At the faang I worked at, some small portion of servers ran the sanitizers in prod, so you’re not reliant on test coverage nearly so much for catching rare issues.

All of these tools and we still have exploitable buffer overflows in 2022. So either the tools are not working, people are not using them or they are using them but can simply ignore critical warnings.

Your milage may vary, but I think Rust offers a well considered step into the right direction. Stupid and dangerous code of the kind every developer will produce once in a while just won't compile in Rust. You cannot forget to run a check, you can't hide behind not knowing a tool. You can't ignore the warning of you want a running program.

That is not nothing.

Yeah, but Rust guarantees sanity by design. Sanitizers are a patch, and hence not comprehensive.

Assuming you have a CI suite dedicated to haning all of the above, sure. Meanwhile some of us are staring down the barrel of 30-60 minute compile times for a single configuration. Multiplying that out to compile with ubsan, asan, and running the static analysis tools over it would take probably 5-6x longer, _and_ we likely need to do it all twice to ensure were checking our "shipping" code paths.

It's not feasible to have a developer compile for 3 different platforms in two configurations with and without sanitizers, and core guidelines checkers for every change, and these tools take so long it's a huge cost.

Would you trust someone new to C++ to use these properly? The great thing about Rust is that you get all this and more (Rust's static checks are a lot more watertight) right out of the box without any configuration.

They painted it like reality, though, no?

You seem to paint the landscape as full of tools and imply that they're used. Either they're insufficient or they're often under utilized, simply due to the number of bugs we see. No?

I've written quite a bit of production code in C++, Python and Rust, and currently work on a hybrid Rust/Python system. Here's my experience:

- C++ is an unusually large language. And it has many historic footguns, requiring a higher level of vigilance and code review. If I were starting a brand new project today, I wouldn't try to build a team of C++ programmers.

- Untyped Python becomes more difficult to refactor and maintain once you reach 50k to 80k lines on a group project. Typed Python, however, scales nicely beyond this size.

- Rust is a "medium-sized" language. It requires developers to learn more than Go or Python does, but less than C++. And Rust has far fewer traps for the unwary and the reckless than C++. Rust's tooling is also very good in many areas.

- It's tempting to split a project into a fast "core" language, and high-level "glue" language. There are real advantages to this. (Which is why I've done it on one recent project!) But this also comes with costs: everyone needs to be fairly good at two languages, and switch back and forth. And you pay a tax at the boundary.

If I were building a brand new database (and a team to maintain it), I'd actually be strongly tempted to use Rust exclusively. But this is partly because databases rarely have a "business logic" layer that changes constantly, so there's less need for a high level scripting language.

But with a different team or different constraints, C++ could also be the right choice.

Especially business logic should be taken in the firm grip of static compile time guarantees that the hand of a strong type system delivers. Even more so if it changes constantly! Refactoring without fear.

Only software that does not have to run correctly (prototypes, personal hobby projects) can get away with a non-static type system.

When I have to pick a tool and I see it is written in Python I will have a look for an alternative if possible. Because I know it will have many bugs: some known, lots hidden.

> Typed Python, however, scales nicely beyond this size.

Could you say more about what tools and practices make this possible, beyond simply adding type annotations in your code? Asking for a friend.

> It's tempting to split a project into a fast "core" language, and high-level "glue" language.

I did this with C++ and Boost Python back in the day and loved the experience. I wonder if Rust will someday get a high-level language for writing applications and scripts on top of a Rust codebase, like Boost Python for C++ or Tcl for C.

> Could you say more about what tools and practices make this possible, beyond simply adding type annotations in your code? Asking for a friend.

Python with type annotations works really well with type checkers like Mypy, along with LSP servers, and both of those integrate with most development environments.

Using a Python-oriented IDE like Pycharm with type annotated Python also allows for better refactoring options. It reduces the uncertainty and guesswork an IDE's static analyzer must engage in for even basic features you'd take for granted with IDE and statically typed languages.

In practice, developers don't have to keep what can be a massively complex application running in their heads to modify code accurately. A nicely typed project makes it easy to see exactly what types of data get passed around and modified. Before gradual typing, you'd have to backtrack to all of a function's call sites to understand exactly what kind of data it takes and returns. With gradual typing, you can just look at types and rely on Mypy to ensure the right data is passed around.

> I did this with C++ and Boost Python back in the day and loved the experience. I wonder if Rust will someday get a high-level language for writing applications and scripts on top of a Rust codebase, like Boost Python for C++ or Tcl for C.

I haven't used Boost Python, but there are some options for Rust and Python that work well and seem to suit this use case like PyO3.

"But this also comes with costs: everyone needs to be fairly good at two languages, and switch back and forth."

Why does everyone needs to be good at both languages? You can seperate and have the core people writing efficient low level code - and you have higher level scripting/gluing code.

You will have Conway's law in your codebase. Coordination between teams is hard, so teams will prefer to implement features entirely in their language, even where that is technically suboptimal.

You will get hot loops in Python, because a Rust programmer wasn't around, and Rust programmers implementing whole complex business logic in Rust behind a single `do_it()` Python call.

Communication and coordination is surely hard and things like that surely happen, but this is why project management exist.

If it is doing things right, then the rust people don't do complex buisness logic, because it is not assigned to them and they would not even have the details.

And if the python people were too eager and have core stuff implemented and it is affecting performance, than you can always reimplement it low level.

It all depends on the project of course, of what would be the best mix.

A brand new project doesn't need legacy C++ footguns. It can use modern C++.

The part of Python (usually) is that you don't need to be "good at it" it you aren't trying to write super polymorphic core that runs super efficient computations like scipy. If you have a fast core engine for the innner loop, a slow Python management layer is plenty fast.

It takes longer to learn how to use C++ to the same level of proficiency and correctness compared to Rust, in my experience. It's harder to write an incorrect program in Rust.

What are the main correctness risks in C++ if you just never use a raw pointer?

I write c++ every day. Theres no such thing as "just never use a raw pointer", as existing code will use pointers, libraries become pointers, and there's no "non-owning" equivalent of a unique pointer. Even if there was:

- lifetimes are hard. The core guidelines recommend using span and string view, but correct usage of those isnt straightforward:

    std::string_view make(const std::string& in)
    {
        return in;
    }

Is not safe, at all. Span is also super dangerous:

    std::vector<int> vec = {1, 2};
    std::span<int> sp = vec;
    vec.push-back(1);

Lambda capture semantics are another area where you might end up surprised by the behaviour too, (and by surprised, I mean you'll have memory issues).

Then there's all the normal stuff that still exists like slicing, lack of bounds checking, resource leaks because of improper inheritance use, data races, use-after-free issues. Sure these can be caught by a sanitizer with enough effort, but they still exist.

I don't know about "main", but like, you don't need raw pointers to have UB. uniq_ptr is nullptr after you move it.

And even then, my understanding is that raw pointers are still intended to be used in Modern C++: they're there for when you don't want to transfer ownership.

One thing off the top of my head, from experience:

std::string s(s);

To be fair, compilers will warn you about this nowadays. But when I converted a C codebase to C++ 20 years ago they didn't.

IIRC, references can also refer to de-allocated memory. Also, if you don't pass-by-reference or pointer, you can literally "slice" the dynamic doohickies off your instance so your AlbinoCat behaves like a Cat because all that extra special stuff is gone as far as the function is concerned.

This is just off the top of my head after not working with C++ for 20 years. I'm sure with all the new features it's gained over the past 20 years theres whole new exciting ways to blow your leg off.

how do you observe data you don't own if you never use a raw pointer or reference?

if you use shared ptr for this:

1. shared ptrs aren't made for this use case, they're for shared ownership 2. why using C++ at all if it means reducing its performance to an (atomatocally) performance counted language? Rust allows for much more performance with its safe borrowed references.

That also not what the CPP core guidelines, that are supposed to define modern C++, prescribe: For general use, take T* or T& arguments rather than smart pointers[0]. This rule incurs the risk of use after free if the returned reference is kept for too long by the caller. The rules for reference validity are non trivial (eg you returned a reference to an item of a vector, if you push into your vector you might invalidate the reference), so this is a significant source of bugs even in modern C++.

if you don't use shared ptr or references/pointers for this i'm curious as to which mechanism you're using to observe non owning data

[0]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines...

Well...

   UNSAFE {   
      // TODO: Verify all the lines, all the time, are ok
      // Just like you do testing, documentation, security and all that
      // ok?
      #include <iostream>
      using namespace std;
      int main() {

         // YOUR CODE
      }   
   }

What are you saying?

Well if the questions is:

> What are the main correctness risks in C++ if you just never use a raw pointer?

All the code on C/C++ IS a correctness "risks". Only constant, manual inspection could(maybe) say otherwise.

What Rust gives is significant reduction of the risks.

I agree, rust has a difficult learning curve. I’ve often heard at least a year is required to really feel confident.

People really vary, it felt natural to me almost immediately but Rust's surveys indicate that some noticeable proportion of respondents report not being comfortable after more than a year.

This is seen as an important thing to improve because of the slogan "A language empowering everyone to build reliable and efficient software". If Rust isn't empowering everyone because say, 10% of people who try to pick it up can't get anywhere that's not good enough.

It takes a relatively short time to be proficient enough to make useful contributions, maybe half a year to a year to be confident. You can give an experienced developer the Rust book and have them contributing to a Rust codebase quickly.

For me it was ~2 weeks to get out of "learning mode" and into actually writing production code. 3 months to feel proficient. Maybe a year to feel that I had reached parity of fluency with languages I already knew.

6 months is what I heard. I'm currently at ~2 and 6 sounds like a pretty good estimate.

> Is Rust such a straight-forward language that people new to the language can write a very performant programme

Yes. At least that was my experience coming to Rust from a primarily JavaScript background. My first Rust program wasn't as fast as it could have been (I know a lot more about optimising programs now than I did then), but it was still ~30x faster than the JavaScript and Pythons versions my company had attempted to write first. That's without putting all that much effort into optimising it.

>a very performant programme

You might need to define what is "very performant".

I come from C++ performance work side in games. Typical C++ is ~10 times slower than optimized C++. And optimized C++ is sometimes ~2-10 times slower than what game hardware can do. What limits many projects is that cost of going to 'next level' of performance is also nonlinear. Where is on this scale Rust code written by new to the language?

The footgun-to-appropriate-feature ratio is higher with C++ than Rust. Rust also has some excellent Python integration options that are relatively easy to use.

I'm 3 weeks into Rust, and I've found it easy to get over an initial hump. I think where it becomes difficult is in thinking about smart pointers like Rc, RefCell, Arc, etc -- stuff I haven't even encountered yet. For things like certain popular leetcode questions, these concepts become more valuable.

> What stood out for me in the article is him saying that it's difficult getting developers with experience in both Python and C++.

More like they had difficulty finding cheap experienced c++/python devs.

Reply on news.ycombinator.com