Rust for the small things? but what about Python?
dataengineeringcentral.substack.com
dataengineeringcentral.substack.com
Has it? Because this is literally the first I've heard anyone claim (or claim that others have claimed) Python is on a downward trajectory. If anything it's become the de facto standard language for anyone doing anything, other than low-level hardware programming; from data science glue code to web applications to one-off scripts to backend pipelines to command line tools, it seems like "Python" is the default answer these days.
It's not the biggest of deals (and I doubt it will significantly hurt Python adoption for niches like data science which have complex installation requirements anyway). But it definitely keeps me from using Python for "quick scripts" which it should otherwise be a good choice for. And it's frankly embarrassing that a supposedly modern language is barely doing better than C in this regard (ok, it's quite a bit better than C, but still a way behind what I would expect).
Having said that, stability is also a big deal, so maybe you're right. Perhaps both are a big deal.
The Python leadership has refused to take a stance on picking a winner. The power vacuum has lead to multiple competing tools, which all do things a little differently.
I have taught Python to a few people and the initial ramp up is embarrassing. I cannot point to a guide on Python.org that says This Is The Way(TM). Instead, I have to give an opinionated workflow full of caveats on how to setup an environment, because there is no “correct” way to get started.
I really do not care who wins. I have had to adopt and transition many packages over the years. Just pick something.
https://peps.python.org/pep-0387/#making-incompatible-change...
That being said, you can use AssemblyScript, which offers a “type hints that optimize” approach. Unlike TypeScript, AssemblyScript is compiled to WebAssembly and leverages type information.
Under the hood, V8 performs its own shape analysis on objects to optimize performance. It’s quite effective and can handle a lot of optimization scenarios, though it would be interesting if V8 could use TypeScript’s type information to pre-seed the optimizer with known object shapes (it does not currently).
Like Flask and other backend stuff, or has someone figured out how to turn Python into a WASM and we're all doomed?
Python is great for something I will use myself, but not so great for when I want someone else to use my code.
If I give somebody a binary, I have to compile this binary for their operating system and architecture. If they don't know how to program they'll be confused as to why they cannot share this program with others.
If I share somebody a python script, they can still execute it as a program as long as a shebang is set. They can use this script on any operating system and can open it with at text editor to view it's inner workings.
This makes scripting languages like Python much more accessible for single-file scripts.
As long as they have the right version of Python and all of your program's dependencies installed, which they most likely won't.
> They can use this script on any operating system and can open it with at text editor to view it's inner workings.
Someone who doesn't know how to program most likely isn't going to be using multiple operating systems and isn't going to want to open the script in a text editor to view its inner workings.
I agree with the parent comment that distributing binaries is way more seamless when your users are not very technical.
The venv created for this is ephemereal (it can not be if you want), so you don't need to keep in mind cleaning up stuff and so. Also uv is really fast in creating the venv and installing whatever is needed. Coming from using plain pip and venvs (and having pain setting up different python version interpreters for projects), and poetry just after that, I am pretty happy with the improvements.
The equivalent things in Go, Rust, etc are a breath of fresh air for sure
I'm not familiar with Python, but I presume its similar to Ruby, where besides trivial scripts, this is false - as soon as there are libraries required, you need to find a way to handle them, which in some cases may include handling multiple envs (I write and distributes such scripts).
Compiling for multiple environments in a controlled environment is by far much more convenient than debugging scripts failing on end users' systems.
Since when has windows come with python installed and worked with a shebang?
Mac OS doesn't come with python installed and clicking it won't run it.
So anyone " confused as to why they cannot share this program with others." isn't going to be able to use your python script.
As they may not have same python version, may have different python libraries (version)
It's not simple to run a python script specially one which uses libraries (let's be honest how often your core is not using those popular libraries)
Not if you use any libraries.
Considering how widespread Python is it seems like solving the one-executable problem for Python should be easier than solving Rust’s problem (heavy compilation). Like with a wrapper or something.
That someone is too lazy to do so is not Python's problem, that's their problem.
It's much simple to write scripts in Go.
Here's an example; https://github.com/zerocorebeta/bashlike
It would be a different matter entirely if that piece of code is executed more frequently and is also taking a lot of time as I could save both computing resources and money
That spread in performance is not realistic though. Of course rust is that much faster, I have no doubt that you could 100x the performance of equivalent python, the problem is that it's an imaginary scenario. Nobody is writing a tight loop in python, you're delegating to pandas/polars, SQL, external calls or what have you. Nobody cares that the code that initializes a call to a 10-minute training run of Xgboost takes 0.001 seconds instead of 0.1 seconds. Nobody cares if you fearlessly optimize the 7 lines of code to connect to a database, whose runtime is dwarfed by a SQL query over the wire.
I don't want to come off as negative towards rust, I do think it's a great language, I'm just really perplexed when people try to use it in contexts that are far removed from its sweet spot.
Less Code != less buggy or more stable code, it just means more implicit code. I contend you spend way more time after release debugging runtime issues or patching random edge cases that are just completely eliminated in typed languages, or deploy/env issues that are eliminated in languages that produce a single binary.
Developer efficiency should include your support time after the writing code.
1. Dependency management is s godsent compared to Python. With Rust I'm confident that I'll be able to pull the code on a new machine and just do `cargo build` and it will work. I'd like to use a lot of curses to describe Python here.
2. Python works well if you can fit everything in your head. But 5 years for now it's scary to make even smaller changes in a Python codebase. With Rust you'll get much more support from the compiler, wether it be refactoring, squashing bugs, or adding features.
So in the long run I prefer Rust.
- Very good, fast language for PoC, even for low level programming projects such as compilers;
- Very easy to setup in a new VM - no weird bash scripts, no complex package download, no need to change a hidden configuration file according to a 10 year old reply of a 20 year old SO post;
- Very easy to run - again no need to touch anything, just python something;
- Very good integration in VSCode;
- Virtual env is a bless for multiple PoCs and is very easy to spin up even for people like me who don't work in terminals very often;
As someone who just want to write some code without understanding a million tools, this is a blessing.
lines = Path("in.txt").read_text().splitlines()
trimmed = "\n".join(lines[1: -1])
Path("out.txt").write_text(trimmed)
Granted the example in the article has advantages (like not loading the full file at once) if you want something more permanent, opposed to a quick script ran once/occasionally.* Does speed and safety matter in every application (probably not)
* Developer efficiency matters
———-
Why can’t have everything in a safe and fast language?
Rust and C++ are tricky (in very different ways) due to the lack of GC.
However, there are many languages that are as productive as Python (maybe more so, if you factor in types and functional programming) yet execute 10x faster.
Would they be just as readable, though? Especially for newcomers that are not yet that familiar with programming?
Wait, is either A or B Hermitian? If so we can do better. Is either of them symmetric? Is either of them triangular? Where do you want to allocate them? Do you want to overwrite either of them or you want a new matrix? If so, how are you going to allocate that memory? Would you like to fuse-multiply-add with your order?
Python's answer is "shut up nerd, just A@B". Necessarily you are going to lose some control over the execution.
Different languages take different tradeoffs on the "simple specification vs. detailed control over execution" scale.
The weird iteration can be replaced with Itertools with_position method and filtering on Position::Middle. The python code counts the total lines by reading the file once and then iterates the file again using the count. This would be possible in the Rust approach too and would look mostly the same.
[1]: https://doc.rust-lang.org/stable/std/fs/struct.File.html#met...
[2]: https://docs.rs/itertools/0.11.0/itertools/trait.Itertools.h...
As with any endeavor, knowing your tools helps most tasks. This is what the example looks like with full error handling and a fairly succinct yet fast approach.
use std::{
fs::File,
io::{BufRead, BufReader, BufWriter, Write},
time::Instant
};
use color_eyre::{eyre::WrapErr, Result};
use itertools::{Itertools, Position};
fn main() -> Result<()> {
color_eyre::install()?;
let start = Instant::now();
let path = "foo/bar/baz.txt";
let tempfile = format!("{path}.tmp");
let input = File::open(path).wrap_err(format!("Failed to open file: {path}"))?;
let output = File::create(tempfile).wrap_err("Failed to create output file: {tempfile}")?;
let reader = BufReader::new(input);
let mut writer = BufWriter::new(output);
for (_, line) in reader
.lines()
.with_position()
.filter(|(position, _)| *position == Position::Middle)
{
let line = line.wrap_err("Failed to read line")?;
writeln!(writer, "{line}").wrap_err("Failed to write line")?;
}
println!("Elapsed: {:?}", start.elapsed());
Ok(())
}
As a Rust dev, I find that marginally easier to grok than the python code, probably due mainly to familiarity. But I can see why the python code would be more easier for a python dev.My rust specific dev experience is ~18 months. I presume the OP's experience in Python is probably equal or more than this.
But there are many languages in the world, and there are some that are as productive as Python, yet execute much faster.
I'd like some recommendations for this from the community. As productive as Python but faster execution is the kind of sweet spot I want to explore and learn. Any specific examples or recommendations for such programming languages?
zig and nim for tries to feel like a scripting language while still being fast.
Personally I like clojure for such things.
The type system uses a lattice, which can be a bit unfamiliar, but once you get it under your belt, it's the jam. I highly recommend getting to know it.
Small example of Nim code (from relevant article [2]):
var gc = 0
var total = 0
for line in lines("orthocoronavirinae.fasta"):
if line[0] == '>': # ignore comment lines
continue
for letter in line:
if letter == 'C' or letter == 'G':
gc += 1
total += 1
echo(gc / total)
[0] - https://nim-lang.org
[1] - https://github.com/nim-lang/Nim/wiki/Nim-for-Python-Programmers
[2] - https://benjamindlee.com/posts/2021/why-i-use-nim-instead-of-python-for-data-processing/Once you've got a repl up and running the JVM launch time isn't an issue and repl driven development lends itself well to data munging/exploration
Clojure's data centricity (functional, everything's a sequence, strong data manipulation std lib) really makes it a dream for these quick and dirty data transformation/exploration jobs
If you want to get fancy, https://github.com/nextjournal/clerk (notebook interface) and https://github.com/techascent/tech.ml.dataset (data frame library) will get you there
Python isn't a particularly good force multiplier purely as a language these days. It was very wise to not go full OOP, so we are spared from the truly horrible, but python is basically C with lists and new syntax as most people use it.
It is significantly faster than Python but has a similar vibe.
That being said, performance isn't really an aspect of the language but of the compiler and runtime. PyPy for example, has a Jit compiler.
I think Python is easier to learn than many languages - definitely its competitors at the time Perl and C++ - but I don't see why it is any easier than BASIC, Java, Clojure, Scheme, F#, JavaScript...
"Easy to learn" is difficult to define anyway. Is chess easy to learn? Any child can learn the rules of chess in a few minutes, but it doesn't make them good at chess.
Assembly is like the rules of chess (assuming RISC at least). Super simple, but you still need to learn how to do anything useful with it.
The Python book on my shelf is about 4 times as thick as K&R. So by that (admittedly rather silly) metric, Python is significantly harder to learn.
One might say Python being more expressive makes it easier, but nothing is more expressive than languages like Perl and awk yet they aren't considered "easy".
We could also consider the principle of least astonishment. Python has some surprises: semantic whitespace being the obvious onev and others like a trailing comma creating a tuple. It used to C type languages were "hard" due to what happens if you miss a semicolon etc, but modern IDEs have pretty much made it irrelevant. Python's surprises are still there.
So yeah I think Python sits somewhere with Scheme as being easy enough to get to the point of being able to do useful computing. But there's still tons to learn that any programmer has to learn regardless of language.
I'm sure a Rust version with counting lines won't have any performance advantage... and is easy to write and quite nice to read.
sed 1d
Unnecessarily rude.