Rust: Not So Great For Codec Implementing
codecs.multimedia.cx
codecs.multimedia.cx
For me, the biggest advantages over C/C++ were:
1. Rust's compile-time checks, run-time checks and fuzzing tools make it much easier to write secure code. Since decoders are a notorious source of bad security bugs, this is a big plus in my book.
2. Rust's dependency management makes it rather pleasant to rely on 3rd-party libraries. The combination of an immutable package repository and semver makes it easy to trust that things won't break. And in my experience, the number of 3rd-party libraries is relatively small compared to more popular languages, but the quality tends to be fairly high (especially relative to npm).
3. The tooling is surprisingly nice for a young language. cargo has solid defaults, Visual Studio Code provides auto-completion and tool tips, and there are good libraries for basic logging, argument parsing, etc.
4. Rust makes it relatively easy to write fast code, as long as you use references and slices when appropriate.
Downsides include the learning curve (about a week or two for a C++ programmer to start feeling semi-comfortable), and slower compile times for large modules or ones using lots of parameterized types. My coworkers will write Rust, but they tend to say things like, "Rust is intense."
Honestly after a ~1yr of embedded C co-op/intership experience, I'm familiar enough but not too entrenched to say that C is not that great for embedded/systems. When you're dealing at the hardware architecture level you need more detail than C (or any PL) can reasonably provide.
C doesn't "let you do what you mean", it has no knowledge of special registers, interrupts, timers, DMA, etc. Companies have coped with a slew of macros that are just ugly to write with and make testing much more difficult than need be. If the language had actual support for embedded you'd see support for architecture strictly as libraries (which may be possible in C but certainly not ergonomic or supported by the culture around the language). Library architectures would make writing simulators and embedded unit tests __MUCH__ simpler.
Not to mention the minefield that is the undefined sections of the C spec. A lot of people "mean" for an integer to rollover, but that's undefined and the C spec doesn't care about what you "mean". [1]
Just today I meant for a constant lookup table not to be overwritten by the stack (I had plenty of unused RAM), but unfortunately C doesn't care what I meant. I had to dig around and find an odd macro to jam that data into program space--effectively hiring some muscle (gcc) to break the spec so it behaved the way I wanted. [2] I run into this sort of thing all the time. I know the C spec. I know my hardware. C doesn't care and needs to be beaten up. The only real benefit of C in these situations is that it's a huge sissy and people are really good at beating the piss out of it now.
I'm not very familiar with rust, but if it makes an honest effort to be ergonomic for embedded (might need to be forked). It will eventually crush C.
[1]: https://blog.regehr.org/archives/213 [2]: https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Named-Address-S...
There are constant improvements but I wouldn't say we're there yet. I think we will be though, embedded is something a lot of folks care about.
Has anyone on the rust team tried reaching out to get input/contributions from companies who are heavily invested in the embedded world? I know xilinx pours a lot of effort into language research, they do seem like an very proprietary company though.
Really, that's all. If it was possible to build C++ classes in C or assembly, it would be done. But it's not because C++ doesn't have the kind of ABI (or any) that would allow one to do that.
Of course, in practice C only gets you 95% of the structure layout functionality you need, as the rules for packing structures and bitfields(!) aren't sufficiently nailed down. Still, 95% out of the box is pretty good, and with some care you can get 100%. This matters when writing, e.g., drivers, but also codecs for another example.
Any systems programming language aiming to replace C has to provide support for a C-like ABI.
I'm not praising C here, mind you. It can't be replaced (with Rust or similar) quickly enough.
You would be surprised, but Object-oriented programming has been done in C many times... and even before C++ existed.
On of the most infuriating things about fancy languages is the way they all have their own ABIs. I mean it's nice that they support special features through those ABIs, but standardized vtables and dictionaries would take us a long way.
And Kudos to Microsoft for having a second crack at getting that right with .Net.
Still, I think a public and stable ABI with fine-grained control of binary elements is a critical aspect of a systems programming language.
Also, the linker contributes a lot to making a systems language. E.g., being able to map globals to specific physical addresses. And, of course, either an ABI has to be dead trivial from a linker's perspective, of the linker has to be very intimate with it (think of symbol mangling). The linkers we use all evolved in a C world, and it shows. C w/ a really good link-editor and RTLD (I'm thinking of Illumos') is a much more advanced beast than C with plain old static linking. Even the ABI aspects of dynamic linking are simple, public, and stable.
A great systems programming language has to have a great linker story as well.
IBM did it first with TIMI for AS/400.
Also Microsoft is now in their third crack at getting it right with UWP (based on COM).
IBM's mainframe OS/400, nowadays known as IBM i supports OO Assembly.
MASM also used to have OO Macros.
See, no tedious rules to remember, you just have to understand how computers work. But the C standards call such things "undefined behaviour" rather than "platform specific" behaviour, and then try to pretend that the compiler can abstract the underlying machine away. That is they pretend programmers can understand the standard, and don't need to know how computers work.
The result is a maze of arbitrary - but historically rational - rules about when the compiler has to do something sane, and when it is allowed to do whatever the hell it likes to squeeze some micro-improvement from a benchmark.
Which is a nice way of saying that sometimes they'll decide to elide chunks of code because they would only be reachable because of undefined behavior, and if that elided code happened to specifically be checking for and handling that undefined case as an error, too bad.[1] :/
This is the major struggle with abstraction. We want to remove the burden of knowing the ins and outs of the target architecture. Inevitably, we create trouble and fall on our faces when it turns out that the hardware is still in fact there and doesn't like when we ignore it.
It's really an impossible problem. One can't account for every architecture when designing a language. Likewise, one can't feasibly remember the details of every architecture while programming. Honestly I'd be interested to see some tools that approach the problem from a direction other than maximum portability. Not that I think they'd be popular or "good".
Where safety is more relevant than maximum portability.
Everything that isn't really portable is marked as explicit language extension or unsafe construct.
One might complain that it leads to language dialects, but the same is true for C, where certain semantics depend on the compiler and even change between versions.
Now unspecified values are tricky again. They can propagate their unspecifiedness and can be different each time you look at them. x == x can be both true and false and is actually unspecified again. If your compiler is using this, things can get insane. I think here the definitions of the C standard would need to be changed a bit.
Also I don't see why signed integer overflow cannot be implementation-defined behavior.
Is it worth a few developer days worth of work to track down a hard to repro bug that only happens with hard to debug optimizations enabled?
GCC supports -fwrapv and -fno-strict-overflow; and I think clang supports both, too. I've never cared to use them because I only rarely use signed types. But some projects and programmers use those options habitually.
AFAIU, Rust panics by default on signed overflow. And even if it wraps, that's not unequivocally better. Unlike with enforced buffer boundary constraints, neither is clearly better than what C does. Arithmetic overflow is a common and serious issue in just about every language. Short of a compile-time constraint or diagnostic that triggers if the compiler cannot prove overflow is either explicitly checked or benign (that is, a negative number is no worse than a large positive number in the context of how the value is used), there's no obvious solution that really forecloses most exploit opportunities across the board.
Because so much code, regardless of language, has some unchecked signed integer overflow bug, if you panic you make it easy to DoS an application. And a DoS can sometimes turn into an exploit when you're dealing with cooperating processes. For example, you occasionally see bugs where an authentication routine fails open instead of failing closed when the authenticator is unreachable.
If you silently wrap signed overflow, all of a sudden the value is in a set (negative numbers) that might be completely unexpected. Even in so-called memory safe languages negative indices can leak sensitive information or erroneously select privileged state. For example, in some languages -1 selects the last element of an array. You can check for negative values explicitly, but multiplicative overflow can wrap around to positive numbers, which is no better than using an unsigned type; a check for a negative values is typically redundant work which adds unnecessary complexity--and unnecessary opportunity for mistakes--relative to sticking to unsigned types.
IMO, signed overflow is the worst option. I just don't see the point. The only three options I like for avoiding arithmetic overflow bugs, depending on language and context, are
1) Check for overflow explicitly (independently from array boundary constraints) and bubble up an error;
2) Carefully rely on unsigned modulo arithmetic;
3) Carefully rely on saturation arithmetic.
IMO the C standard's fault isn't in its refusal to make signed overflow defined or implementation-defined, but in providing neither a standard API for overflow detection, a construct for saturation semantics of integer types, nor a compilation mode to warn about unchecked signed overflow (e.g. something at least as useful as -Wno-strict-overflow in GCC).
Fortunately both GCC and clang have agreed on a standard API for overflow detection. That's something. But unfortunately it'll be years before you can consistently rely on those APIs without worrying about backward compatibility.
Overflow of any integer type is considered a "program error", not undefined behavior. In debug builds, this is required to panic. In builds where it doesn't panic, it's well-defined as two's compliment wrapping.
You can also request explicit wrapping, saturating, etc behavior.
| Type | Bits | sizeof | Range |
+---------------+------+--------+----------------------------------------+
| ... |
| unsigned long | 36 | 4 | 0 to (2^36)-2 (see the following note) |
...
Note: If the CONFORMANCE/TWOSARITH or CONFORMANCE/FULL
compiler keywords are used, the range will be 0 to (2^36)-1.
See the C Compiler Programming Reference Manual Volume 2 for
more information.
-- Section 4.5. Size and Range of C Variables of the Unisys
C Compiler Programming Reference Manual Volume 1.
https://public.support.unisys.com/2200/docs/cp16.0/pdf/78310422-012.pdfI wonder if it still does end-around carry...
rude.
Special registers are just slots at specific memory addresses.
As far as the rest is concerned, not a single language in the world will provide you with primitives for that and make them portable across a thousand architectures with a million different peripherals. That's the role of libraries and OSes with their drivers.
> As far as the rest is concerned, not a single language in the world will provide you with primitives for that and make them portable across a thousand architectures with a million different peripherals.
I'm well aware, that's why I said.
> When you're dealing with the hardware architecture level you need more detail than C (or any PL) can reasonably provide.
A language does not need to provide every hardware primitive, just realize that it can't and implement a sane way to support hardware features that don't require abusing a cumbersome macro system and heavy compiler modification.
> That's the role of libraries and OSes with their drivers.
A ridiculous amount of vulnerabilities and wasted effort are sunk into operating systems and their drivers because of the garbage state of tooling for low level work. It's easy to make statements such as yours until it comes time to write/maintain a multi million line kernel and a only the lord knows how many line driver/subsystem fleet.
Kernel/driver devs do their best to make due, but the situation is far from ideal.
The information has to somehow be passed to the compiler, your options are: compiler specific features (see gcc attributes, etc), language level (see the fun that becomes with explicit_memset vs. memset) or a really fancy IR that can hold these types of requirements.
I'm confused in that, it's true, I've seen the macro system abused. But for working with things like memory mapped registers and interrupts you don't _need_ to abuse it. It actually looks really nice with with memory-mapped structures that ply on the register structure. Maybe I'm just missing something.
I think the only reason C is very popular for systems programming is that it allows you to do most of what you could do in assembler, but in an easier-to-work way. So basically C is an "acceptable, easier-to-use assembler", but far from ideal, because of the reasons you point out.
Honestly, the popularity of C itself is due to the success of UNIX, not due to any particular quality. On the other hand, the K&R "The C Programming Language" book is a classic, a well written book which surely does a great job of introducing the language.
But back then in the late 1960s, a new language called ALGOL-68 was specified, back then a very advanced language which had powerful features even for 2017. The problem is that nobody dared to implement the full language. CPL was a stripped down implementation of Algol-68, this was then stripped down even more to BCPL; then it was extremely stripped down to B (only one datatype), then Dennis Ritchie kept most of the syntax and added other data types and C was born. C was never designed from the ground up to be the best systems programming language possible; only to be able to work for Dennis Ritchie & friends' "toy" operating system, UNIX, a very stripped-down MULTICS.
I think the "UNIX-Haters Handbook" has a section devoted to C criticism.
On small embedded computers, I think C is popular because it's so easy to build something that resembles a C compiler. The fact is, undergrad compiler classes almost build C compilers (some universities might actually go all the way.) You can almost build a yacc grammar that will read C and emit assembly. The small computer embedded vendors have a difficult problem, they need to provide tooling and they don't usually have giant piles of developers to sell to. So they dust off some abandoned "C compiler" they found in the gutter, add their hardware to it and call it good. At least that's my theory and I've seen some rough "compiler work" from some pretty substantial hardware vendors. In fact, if you're running a 16bit or smaller part or a weird part, I have yet to see a decent C compiler from a hardware vendor.
I'm hoping the compilation world progresses enough with efforts such as LLVM to break our chains to crufty languages.
I have looked at a lot of C parsers and compilers and have written a C to Lisp transpiler and this statement is ludicrously wrong. You cannot even parse C code with yacc because of typedefs. The grammar of C is context-sensitive. And this is after the C code has been preprocessed, something that also cannot be done with yacc.
C compilers are easy to port to different machines, but that is because C is a very poor language in terms of features and control flow constructs. For the limited amount of things that C gives you as a language it also comes with a huge amount of complicated baggage when it comes to implementation and corner cases.
I beg to differ. C was a fairly obscure language until MS-DOS came out. C turned out to be ideal for programming on DOS, and DOS programming was far and away the most programmed system in the world for a decade and a half.
MS-DOS also made C++ into a major language (via Zortech C++). When ZTC++ came out the penetration and popularity of C++ went through the roof.
(Yes, I'm tooting my own horn a bit here. But I honestly believe it is ZTC++ that got C++ its critical mass.)
MS-DOS made C and C++ into the juggernauts they became.
After the success of ZTC++ and TC++, Microsoft changed direction and decided to develop a C++ compiler, too. I heard (but was never able to confirm) that Microsoft had earlier been developing their own object oriented extensions to C called C*.
I could even say something similar about turbo c vs turbo c++, but that was more due to the language issues with c++ vs c.
I only started using TP with version 6, so all I know is OOP TP...
As I had to give a class on OOP to fellow students at the technical school as exchange for having access to Turbo Pascal 5.5.
Never was a big fan of some of the Object Pascal changes made by Delphi's class model, specially the fact that it was a kind of lost the opportunity to introduce RC alongside those changes.
Which eventually lead to the schizophrenic model that RC would only exist for COM based classes, but not others.
I maintained the Delphi compiler front end for 6 years or so, adding closures, enhanced RTTI, a bunch of work on generics and 64-bit porting, and some other things that ultimately didn't see the light of day. I have a long list of things I don't like about the Delphi language, corner cases you only really become fully aware of when living with the workarounds they force on the codebase.
Overload resolution is an almost wholly unspecified mess, for example. It started out Java-style, but the definition of more specific isn't locked down, and everything from strong typedefs (type TFoo = type TBar;) to closures (MyFunc(methodRef) - do you mean to pass method ref or result of calling method ref), to ranges (are smaller ranges more specific? what about overlapping ranges?)... and don't get me started on all the different string types.
The first time I went through the Turbo Pascal for Windows 1.5 manual I wasn't that happy to see PChar, let alone the other variants that came later.
Although it seems that in today's world, most languages end up with a jungle of string types, for every possible variation of Unicode, ANSI and C ABIs.
I eventually did the full transition from Object Pascal to C++, so I only used the very first versions of Delphi.
So in what concerned Windows development, I ended up moving from Borland to Microsoft compilers when Visual C++ 6.0 was released, which means I lack the experience how later Delphi versions evolved.
On the early days of DOS, what was used the most was IBM-BASIC (and GW-BASIC) for users, assembler for games like Alley Cat, IBM LOGO for teaching, some games were made on FIG-FORTH even
Then borland brought Turbo Pascal which was hugely popular together with MS QuickBasic.
Meanwhile, and we're at 1986-7, a good amount of software was already made for the Unices, in C. I don't think anyone was doing C for DOS back then. My grandpa -a PC nerd- had a huge (200+) collection of diskettes around that era, which now I own, there are all sorts of compilers for the above languages, plus Fortran, Clipper, etc; but no C compiler. Even though he owned the K&R book.
First time i saw use of C on the DOS world was with Borland Turbo C++.
Borland produced Turbo C not to introduce C to the PC, but because C was so dominant on that platform. After all, why would they have done that after being so successful with Turbo Pascal?
* Aztec C
* C86
* Datalight C (mine!)
* DeSmet C
* Eco-C88
* Hot C
* IBM C
* Lattice C
* C Prog Sys
* Let's C
* High C
* Microsoft C
* Mix X
* Toolworks C
* Whitesmiths C
* Wizard C
PC Tech Journal reviewed 8 C compilers for DOS in their Nov-Dec 1983 issue, though I don't have a copy of it.
On the PC, C and C++ only started to be widely adopted as we started to code for Windows 3.x, due to the APIs being available as C instead of interrupts.
At the technical school I was attending, I was the outlier by having had the opportunity to get hold of Borland's Turbo C++ 1.0 for MS-DOS, everyone else couldn't care less.
Most Portuguese and Spanish magazines of the 80's and early 90's were full all imaginable programming languages for C64, ZX Spectrum, Amiga, Atari and MS-DOS. C didn't had a better spot on those articles than the other languages, quite the contrary.
Fellow Iberian HNers are free to correct my experience of those years.
I don't think that any of this has anything to do with the language itself, it's just that it takes time for things to mature. As for the issues outlined in the article - the language is different enough from C and C++ that one just has to accept the fact that you cannot write idiomatic C and C++ in Rust and expect it to be comfortable, performant or safe. However, with Rust, I'd say, that one can achieve 99% of what one can achieve in C today. The only thing that Rust is missing currently is the ability to arbitrarily jump around the call stack due to the way destructors are implemented, but there's a way to mitigate this and it's being worked as far as I was aware.
https://internals.rust-lang.org/t/getting-explicit-simd-on-s...
It still wouldn't compete with really good hand-tuned asm, but it might help reduce the perf/usability tradeoff a bit.
I agree that this needs to be standardized for vector-ey crates to be able to talk to each other seamlessly. Otherwise we'll end up with a rerun of the C++ strings fiasco, with char* and wchar_t* and std::string and std::wstring and BSTR and _bstr_t and CString and CComBSTR and QString and GString and...
People can and do write vector in libraries right now, usually using existing compiler support that gives them guarantees about SIMD.
AKA, its completely possible, and there are quite a number of C++ libraries that make vectors look like native types complete with long lists of global operator overloads for interaction with other base types. Generally these libraries are just thin wrappers around inline assembly or intel intrinsics for SSE/AVX and generally don't bring any form of OO syntax into the picture.
That said, even with C++ the libraries tend to fall down a bit when it comes to individual element operation/masking because the closest method is generally using array syntax for the elements which limits the operations to a single element in the vector at a time. Which means you end up with OO method calling syntax for element masking (or creative solutions with operator() )
(Also, Rust has pretty transparent support in the same manner also without OO overhead, but differs slightly because methods can and often do use pass-by-value. This is the distinction I was drawing.)
OP was specifically recommending operator overloading on a plain old struct, which can be done without declaring a class. Indeed, operator overloads can be declared as global functions in C++. The this pointer doesn't enter in to it at all in that case.
The only "downgrade" I made was saying that it is only "almost" the default in C++, versus the other two where they are completely the default.
To be clear, like C and Rust, C++ has great semantic attributes for this sort of thing:
- classes/structs that don't require allocation/pointers
- precise control over pass-by-value (for everything except the `this` pointer of methods)
- pervasive static dispatch of methods/functions (including operators, which can be considered to be method/function with an unusual name and call syntax)
The only downside, and the reason I said "almost" (which is what the C++ETF (C++ Evangelism Task Force) seems to be up in arms about), is methods are what most people reach for by default and so the `this` pointer comes into play. But as you point out, and as I implied in my original comment, this isn't required, just the default.
Standardization. Optimal performance. Better Syntax.
Bounds checks can be eliminated and code can be vectorized if the optimizer can prove it; explicit SIMD is useful for the cases where it can't.
And, that is assuming that the autovectoriser is able to synthesise the desired instructions, e.g. I believe SSE2's packssdw & packusdw ("pack with signed/unsigned saturation") and pmaddwd ("multiply and add packed integers") are useful in a JPEG codec but I find it extremely unlikely that any compiler will autovectorise to them.
Who said anything about doing it safely? You can elide the bounds checks explicitly with calls to get_unchecked (or whatever) using unsafe.
> How would Rust handle SIMD differences when trying to generate specific code as you would in C?
Please be more specific. This question is so broad that it's impossible to answer. At some levels, this is the responsibility of the code generator (i.e., LLVM). At other levels, it's the responsibility of the programmer to write code that checks what the current CPU supports, and then call the correct code. Both Clang and gcc have support for the former using conditional compilation, and both Clang and gcc have support for the latter by annotating specific function definitions with specific target features. In the case of the latter, it can be UB to call those functions on CPUs that don't support those features. (Most often the worse that will happen is a SIGILL, but if you somehow muck of the ABIs between functions, then you're in for some pain.) The plan for Rust is to basically do what Clang does.
The question of safety in Rust and SIMD is a completely different story from auto-vectorization. Figuring out how to make calling arbitrary vendor intrinsics safe is an open question that we probably won't be able to solve in the immediate future, so we'll make it unsafe to call them.
And even that is all completely orthogonal to a nice platform independent SIMD API (like you might find in Javascript's support for SIMD[1]), since most of that surface area is handled by LLVM and we should be able to enable using SIMD at that level in safe Rust.
And all of that is still completely and utterly orthogonal to whether bounds checks are elided. Even with the cross platform abstractions, you still might want to write unsafe code to elide bounds checks when copying data from a slice into a vector in a tight loop.
[1] - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
I've seen it used for various video/photo processing operations.
If you want to use vectors explicitly, you can do so using intrinsics for most vector architectures.
Assuming that's true, you want to maximise performance by using as much of the parallelism that vectors give you. So if you're dealing with [4 x int32], on a 128 bit vector ISA you would be fully utilising your vector registers, but moving to say AVX-512 you're now only using 1/4 of your potential parallelism.
Your architecture independent vector types would have to target the lowest common denominator, and completely defeat the purpose of vectorisation.
How these map to any particular processors resources is not my problem - though the three major vector extensions today all have 4-element vectors. Some support more, but that's not terribly relevant to the math I want to do. A smart compiler could pack multiple small vectors into a wide vector register just like they try to pack multiple scalars in there today.
I am not interested in vectorizing loops. I want to write code like this:
Vec3double position = {5.8,3.9,2.1};
Vec3double velocity = {1.0,0.0,0.0};
double timestep = 0.01;
position += timestep*velocity;
and so on. Yes, I also want the common use case of multiplication of vector and a scalar to be that easy.
Any paint program or graphics library (including font rendering) does a ton of this stuff. So does every 2d or 3d physics engine. Ray tracing. FEA software. CAD. The list of uses for these vector sizes is long and has nothing to do with auto-vectorizing loops. Of course there are plenty of applications where loop vectorization is valuable and I don't want to take anything away from that. I just want built-in support for these common mathematical entities in the base language.
On ARM you have specialised load and store instructions that can de-interleave into vector registers such that register a contains VecType.x and register b has VecType.y etc, but are a bit slower.
If you don't care about SIMD performance then fair enough, but if you care enough about this issue to want the compiler to generate SIMD instructions, you better be willing to change your code to be performant on your particular target because even small changes can impact whether or not it's worth vectoring vs leaving it as scalar code.
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p019...
[0]: https://play.rust-lang.org/?gist=989fe1c05e3e68df89642032743...
allocator: I ran into the same situation and abstracted it with a generic allocator: https://github.com/dropbox/rust-alloc-no-stdlib it's an ugly solution, but it works and can allow for significant perf improvements down the line by bundling all same types together
benchmarks: I simply made a test that printed the time
primitive types: it's easy to write generic functions that do this
macro system: I found to be amazing, but I never used it to compact data types
borrow checker: split_at_mut was super helpful...also core::cmp::replace was useful to taking away pieces of a structure and manipulating them, then putting them back
Though it wasn't all roses: to try to get SSE vectorization I had to convert this 6 line short string matching function https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d...
into this multi hundred line monster with macros and so forth https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d...
but in the end it was as fast as hand-tuned intrinsics in C
I also found myself scared by dependencies, including onto the std library, so I tried to get everything to remain within the core library. This should allow something as low-level as a codec to be used in kernel space or in another place where a custom allocator is needed.
I also found that "rewriting in safe rust from C" was made easy by corrode https://github.com/jameysharp/corrode since C and rust can be interface compatible, it's easy to go one file at a time and turn it into working rust, then safe rust.
https://www.eternum.io/ipfs/QmYrdKpWbHCuNPUrBd6MPFrcTqPBW55w...
https://chrome.google.com/webstore/detail/ipfs-station/kckhg...
It would be easier if Rust had real multidimensional arrays. If the compiler knows about multidimensional arrays, some additional optimizations are possible. For example, if you want to borrow two different rows, that's fine as long as the row subscript is different.
Here's a doc on how to write such a function yourself, and includes the (actual) source code for split_at_mut:
https://doc.rust-lang.org/nomicon/borrow-splitting.html
As others have mentioned, there's nothing special about the standard library in this regard; you can just write the method yourself and it'll work equally well. (The standard library is moderately special in that it gets to use nightly features because dealing with breaking changes is easy for code that literally lives inside Rust's own source repository, but it's not special in terms of its ability to use unsafe code or anything.)
(There's no reason to avoid safe functions that happen to have unsafe implementations. For instance, the standard-library implementation of indexing a slice, as in b"hello"[3], is just a bounds check plus an unsafe dereference.)
With a large single array of 100 elements, but divided into 10 10 element pieces by convention, a slice of elements might be accessed by getting the elements array[10a+b] through array[10a+c]. If we also want a slice from the next segment, that would be array[10(a+1)+b] through array[10(a+1)+c].
With a multidimensional array, of 10 bounds checked segments of 10 elements, we could access by array[a][b] through array[a][c] and array[a+1][b] through array[a+1][c]. If access is bounds checked, or the compiler can prove the values are within the sizes, then the multidimensional version provably has no overlap.
Or am I missing something here?
I was actually thinking of this when I wrote, but the opposite case. A lot of compilers already know about that, and exploit it to their benefit (and sometimes the programmers dismay[1]). Overflow semantics matter here.
That said, it's looks like an interesting area for Rust. I'm sure it's already been discussed to death somewhere in the community. :)
Integrating this sort of thing into the type system "properly" (as in, actually part of the language and doing all of what I think Animats wants) basically means going a long way towards full dependent typing.
[1]: behaviour on invalid code, such as C relying on signed integer overflow, is unspecified/undefined. Languages like, say, Rust and Haskell generally try to guarantee the optimiser never gets invalid code by flagging such instances at compile time, unlike most/all C and C++ compilers.
I've only looked into Rust briefly, but I thought this was the point of unsafe blocks? You can roll your own multi-dimensional array data structure using unsafe blocks if you're certain what you're doing is actually safe.
That said, you can create raw and dereference raw pointers, which lets you to do the above.
It might be nice to have some sort of pattern matching syntax for it, I suppose:
let [ref mut a..16, ref mut b..] = *c;
let (a, b) = c[..16..];Why does the author have to worry about aliasing in the first place? If they're dealing with POD types, there's no memory safety issue implicated by allowing overlapping slices (except possibly when accessing from multiple threads). No-alias guarantees can improve performance in many cases (though rustc currently isn't doing the best job conveying them to LLVM), but not always, and especially in something like a video codec, the author may prefer to manually optimize memory access patterns.
In my own code, I worked around this by using &[Cell<u8>] as an aliasable mutable buffer type. I had a function to cast to it from &mut [u8], as well as functions to read and write integers of various sizes by directly casting the pointer (although this may not be necessary, as LLVM can probably optimize the naive approach of calling .get()/.set() on each Cell individually). Simple enough, and I'm pretty sure it's safe, but I've never seen anyone else use this approach, and I don't know of any crates.io crates that implement it. Maybe I should write one...
There's the external https://crates.io/crates/alias and even an approved RFC for adopting it into std: https://github.com/rust-lang/rust/issues/43038 .
fn split_at_mut(&mut self, mid: usize) -> (&mut str, &mut str) {
// is_char_boundary checks that the index is in [0, .len()]
if self.is_char_boundary(mid) {
let len = self.len();
let ptr = self.as_ptr() as *mut u8;
unsafe {
(from_raw_parts_mut(ptr, mid),
from_raw_parts_mut(ptr.offset(mid as isize), len - mid))
}
} else {
slice_error_fail(self, 0, mid)
}
}
Safe memory management without a garbage collector.
> If Ocaml solves the problem why Rust?
Ocaml doesn't solve every problem any more than Rust does. Rust may solve a subset of the problems people use the listed languages for better than those languages, so it might be beneficial for people to use it in those cases.
> Rust was made to solve the problem Ada solves.
Not all solutions have the same efficiency, or are as easy to understand. Rust aims to enforce safety at compile time to eliminate runtime safety checks where possible. Ada performs many safety checks at runtime, which introduces a performance cost. This does allow Ada to provide some additional safety checks though (such as limiting a type/subtype to a range of values).
I'd be interested to hear the author's reasoning behind this, if it does what they want then why not use it? It's small and well written, so I don't think vetting it should be a problem.
The rest of the article seems quite sensible, that comment just strikes me as a little odd.
With Rust, you really should just use crates. The std is meant to be limited to just the most used code and that which should not change for the sake of keeping the ecosystem stable.
Off topic, but it shouldn't be better than "even Python", because Python has a really, really broken dependency system. Far more so than Java, which has Maven/Gradle which are both infinitely better than the pip/virtualenv disaster.
People complain about things like shading in Maven being complicated. What they might not realize is that pip doesn't even try to address conflicting dependencies, it will just silently give you the wrong version! You ask for A==0.2 and it will give you A==0.1 if another dependency asked for A==0.1 first. And it won't even warn you even though it's straight up broken behavior. Virtualenv makes packaging annoying since it's almost vendoring but not quite. To totally understand the packaging system forces you into the world of eggs, wheels, disutils, conflicting versions, condas, etc.
Sorry for tangent, just thought it was funny you would hold Python up as a standard of dependency excellence when it's probably the worst overall ecosystem of major languages.
To me, this is a clear case of trying to fix a broken system whereas starting from scratch would be much better.
Additionally, for anyone coming from almost all compiled native languages, a native environment like Rust with better dependency management than Python (which, as you say, and in retrospect, is pretty broken) is a bit of a mind screw.
Cargo should even be better than npm.
On the other hand I always asked myself if they couldn't simply use Nix?
Regardless, while it's cool and useful, it's not really actual Windows support.
Didn't take you for a Windows guy, hehe
It's due to video games... but I'm actually enjoying Windows 10.
There are a lot of great things about Python, but dependency management is right there under the GIL on the list of things that are very painful.
I really liked that "dependancy as part of the language and build tool".
Of course, Go still had its own dependancy issues (versionning, availability, ...) that they now work on, but that was a step in a direction I quite liked.
You declare your crate dependencies in a Cargo.toml file. Rust does proper versioning of dependencies so having a separate manifest is desirable. Within your code you declare the existence of a crate via `extern crate` and then you can use it wherever.
> the compiler force you to remove the imports too
it warns.
It's more likely to be a part of RLS than rustfmt.
There's a separate tool called rustfix in development that can apply the suggestions given by the compiler, so it could theoretically prompt for these.
So each new crate added to your project ends up increasing the overall build time, which gets exponentially bad if you happen to add a dependency to a crate than happens to have a big dependency list.
I have a very basic word counting application with a GUI written in Gtk-rs, a fresh build straight out of "git clone" takes a few minutes, mainly thanks to Pango.
That's only a problem the first time you compile though, isn't it?
[0] - I eschew header only libraries, only if there is no technical way around them (e.g. templates).
[1] - I can AOT compile with Excelsior JET, IBM J9, JamaicaVM, PTC, , Oracle Java 9 (Linux x64 for the time being), ...
[2] - I can AOT compile with Mono, NGEN, .NET Native, CoreRT, IL2CPP, ...
Yes, it is a problem when you do a clean build, or when you have common crates across projects, because cargo doesn't have a concept of build cache.
You can try to workaround it by setting all target directories to same one via target-dir in your .cargo/config file, but there is no guarantee that the crate won't get rebuild.
1. not be malicious
2. not write a vulnerability by accident
3. not get their computer infected, their email account hijacked, etc.
4. be wise in transferring ownership
5. not add a dependency with a license incompatible with your project
All the above concerns apply recursively to the dependencies of the dependency.
Bytes is basically just C++'s `std::string` (kind of don't bite my head off people who've memorized the C++17 standard). Its an ARC backed array [1]. This suggests it should be a fairly fundamental abstraction.
Edit: ^^^My C++ is wrong sorry :(
Really I disagree with its purpose. In its immutable, non-threaded safe form you can create the same structure by just borrowing a value. This ofc requires making your peace with the borrow checker and `Cow<'a, T>` copy on write types.
By in large Bytes advertised purpose is network code. And for networking code its really only _super_ useful if your using a Packet Ring in Linux. As jemalloc will not return regularly re-allocated buffers.
Really this is all performance theater. How you manage/architect your socket reads/writes will have an order of magnitude larger effect then what abstraction you _store_ those bytes in once read.
[1] https://carllerche.github.io/bytes/bytes/struct.Bytes.htm
As a developer at a large software company, every dependency that is not part of the language runtime itself is a pain because legal paperwork and evaluation has to be done for each individual component before I can use it / ship it.
NOTE: I am not referring to copying I would be doing, but to the crates thats have little dependencies that should consider including a copy of their little dependency instead via whatever method is appropriate for licensing.
This makes languages like Python, Go, Perl, etc. preferable over languages/projects like node.js, rust, etc. because (at least I) don't generally end up with tens/hundreds of dependencies since the standard library is rich enough for most work.
Additionally, I personally despise have many, small dependencies because it means I have more to think about and manage. Instead of just being able to think about the version of the compiler/standard library I'm using, I have to consider every individual crate.
I'm well aware of why rust chose to make certain tradeoffs with crates, but having numerous pieces of what many of us consider "basic functionality" as a third-party dependency is frustrating. Languages with richer standard libraries have spoiled us all.
I really disagree with that quote. Copying is how you get bugs sticking around in software for all time (for example, doing binary searches in a way that avoids overflow is surprisingly tricky, and the endless copying of naive binary search code is why this bug is so difficult to eradicate). Honestly, that quote is just an excuse to avoid the hard work of making the language ecosystem handle dependencies properly.
> As a developer at a large software company, every dependency that is not part of the language runtime itself is a pain because legal paperwork and evaluation has to be done for each individual component before I can use it / ship it.
How is copying better than dependencies in this regard? You presumably need legal signoff either way.
> Additionally, I personally despise have many, small dependencies because it means I have more to think about and manage. Instead of just being able to think about the version of the compiler/standard library I'm using, I have to consider every individual crate.
This is what the "Rust platform" is designed to address. It is nice to be able to refer to a specific version of the Rust platform, but that doesn't mean you have to give up on the massive ergonomic benefit of the Cargo ecosystem relative to copying and pasting code.
Employees take shortcuts around bureaucracy all the time. Sometimes (often?) that bureaucracy is for legal reasons.
Your argument is the correct moral and legal one. Unfortunately that doesn't always matter. For another example, see the cognitive dissonance many express regarding ad blocking (not to come down entirely on one side of that issue, it's complicated).
If your company gets aquired one day, the code will probably be audited during the due diligence process. If licence violations related to copy-pasting is found, your team will be asked to remove the infringing code and your supervisor may be fired. This happened to my team in the first company I worked for (not the firing part though) : we had a lot of code which was just copy-pasted from lodash and the audit found it.
I wasn't referring to copying that I would do myself, but copying that other crates would do.
That is, if a crate only needs a little bit from a little dependency, then copying it into their crate can make everyone else's life easier (obviously taking licensing into consideration when doing so).
In short, the context here was the bytes crate, which is fairly tiny. If rust is going to insist on not including the bytes crate, or a copy of it, in the standard library, then I would hope others that consume it would consider embedding a snapshot of it into their own crate for their own, private use so that I don't have to worry about it.
I'm well aware there's a fine line here, hence my reference to the Go proverb.
What would you be worrying about?
A component that instead references its dependencies and that have their own release schedule/versions, etc. requires a legal review for that component and each of its dependencies.
This has been true at multiple employers I've worked for, so seems unlikely to be a consideration unique to my current employer.
This is where I actually prefer Go's "vendor" approach to dependencies. It would be great if rust / cargo eventually had the same and more authors adopted it or simply copied their little dependencies instead of having external dependencies on them.
Something like this proposed command, except for crate maintenance instead of distribution:
It's even used for releasing the official Rust tarballs as we now employ crates.io dependencies in the standard library and the compiler.
I think this thread is about copying and pasting code versus using a small library in the Go case, which might be a philosphical difference with Rust.
It might help to point out that vendored crates are compiled from source making the required review process referenced by that poster just as possible with server crates.
Would you rather a flawed, or later deemed incomplete internal solution be implemented and then the language is forced to support it in perpetuity, or would you rather one or more solutions get tried and the best implementation and syntax eventually accepted into core?
EcmaScript can do the same, and finally has[1], but it moves so slowly and has so many competing interests that it seems to take forever for that to happen.
1: https://www.ecma-international.org/ecma-262/8.0/index.html#s...
You can see what happens when you go too far the other way though. C has effectively no basic data structures like strings, lists, hash tables, etc., so anything you interface with has its own idea on how to handle that stuff. Library X might return an array of Things that's NULL terminated. Library Y might return an array of Thangs and a size_t output param. Or like you pointed out in JS, its standard library is full of holes so you get tiny projects that attempt to plug them, or larger projects that try to make JS into a specific kind of language (Underscore), or full on programming languages that transpile to it.
I just don't think the problem is definitively solved though. Personally I think Bytes should be in Rust's stdlib. I think bit and byte manipulation is a fundamental part of a language and there should be a standard way of doing it, especially if there are things like TCP/UDP and hash tables in there. I understand the arguments against; I really like the design of Rust's stdlib, but I feel like there's room for disagreement. That's all I'm saying :)
It's not a problem, in my view.
(This isn't uncommon; it's also true in, say, Elixir: there are a few useful Hex packages owned by the elixir-lang GitHub org itself. And I believe it's true in Haskell as well.)
Unlike a huge stdlib, a "distro"-style SDK is still factored into packages (in Erlang terms, "applications"), that can be included or excluded from any given release of your project. But it's all released monolithically, and comes as one big package. Probably helps a lot with getting legal sign-off for using the relevant packages. I wonder if that's why they (still) do it?
I can't count the number of times we've had problems with the requests library, either because of the huge tree of dependencies (both explicit and implicit) that requests has, and because of some of the assumptions made by requests.
On the other hand, when a bit more time is taken (yes, this means a few lines of boilerplate) and the code uses urllib2, it rarely has to be touched again.
But your larger point still stands.
So even if you don't agree with some of the author's points, his use-case is a good test for the suitability of Rust as a Systems Programming Language.
http://searchfox.org/mozilla-central/rev/bbc1c59e460a27b2092... has some of the gory details.
These kinds of experience reports are so valuable. (I've been following the whole series and they're very interesting.) Even if solutions to these issues do exist, if people can't find them, well that's a problem too.
Most of our users use stable, and want to stick to stable. There's just some use-cases that need features we haven't finished designing yet, such is life.
Contrast this with other languages (C++, Python, Ruby, Go) where I have no idea what features are under development until they hit a stable release. Compare to JavaScript, where stable releases are so slow people built transpilers ;)
This could also be because I'm interested enough in Rust to occasionally land on the first-party discussion forums, of course (probably helped by how open the language development is). I'd say it's also young, except that it now has a release number much larger than 1.0 and doesn't really get to claim that anymore.
I think that's the right assumption, because it's an acknowledgement that the issue actively being worked on to the degree that they have at least a partial solution implemented, as opposed to the "someone is working on that" response language developers sometimes give which usually predates that and could mean someone is reading literature and surveying the field and no code has actually been written.
As you note, because development is so open, nightlies are easily accessible, so people might incorrectly infer that they are being recommended to use them, or that the problem is solved. I think what the devs are attempting to communicate is that it's close, and if you want you can test it out to see what it's like and give feedback. This might be a case of similar groups with slightly different context (lang devs / lang users) interpret a statement differently. From what I've seen, the Rust devs see fairly responsive to communication issues when pointed out, so maybe this will result in some change (or perhaps they are acutely aware of the issue already and instituted changes have not shown benefit, at least yet).
Actually interacting with rust-related people, whatever their capacity (developer, user, etc.) tend to go pretty well; it's just that once in a while a random comment shows up that seems dismissive of actual problems people are reporting.
You're probably right though, it's just that people have different contexts and I'm reading more into things than intended. Again, rust development is extremely open (which is great) and these things just tend to not be visible for other languages, rather than not happening.
Yup, that's absolutely it. I'm not sure of a good, succinct way to imply the former rather than the later...
> it now has a release number much larger than 1.0
We do releases every six weeks, so while it is in some sense, it's also just going to be higher than other languages.
Personally, a sentence saying something to that effect (i.e. showing that it's understood that it doesn't immediately hello just yet) would be sufficient. It would probably get tiring every time nightlies are mentioned, though, but it means people (like me) who randomly land on a thread would know that the problems are acknowledged.
> We do releases every six weeks, so while it is in some sense, it's also just going to be higher than other languages.
Right, but the problem is that the rest of the world has expectations on those numbers (e.g. x.0 is probably buggy, >0.x is a thing external people are expected to be able to use). The particular minor version number doesn't matter as much. That ship has sailed (and probably circumnavigated the world a few times) though.
The biggest problem is probably that I'm comparing it to golang, which was the last 1.x language I learned, and that had a more complete standard library by 1.0. Again, likely the effect of the much more open development model (no baking behind closed doors).
Out of these points they were all for features which are steadily on their way to stable (or future things which are actively being worked on right now).
Very few folks use nightly -- many use it locally for improved developer tooling, but mostly everyone wants their crate to work on stable.
So I wouldn't say it relieves pressure that way.
> Compilation time is too large
Can you try compiling incrementally? https://blog.rust-lang.org/2016/09/08/incremental.html. Might still only be on nightly.
> And, on the similar note, benchmarks.
I agree, profiling as well isn't as full featured as in more mature languages. Clojure, incidentally has great benchmarking due to being on the JVM.
> Also the tuple assignments.
Can't you just do
fn main() {
let (a, b) = (5, 2);
println!("{}, {}", b, a);
}
I think this will be solved with the new non-lexical lifetimes RFC. Also a problem I had when starting, I generally assume referential transparency.
That's a declaration not an assignment, TFA is (somewhat oddly) using the language precisely and fittingly: https://doc.rust-lang.org/reference/expressions.html#assignm...
And there are cases where shadowing is not acceptable e.g. when updating bindings within a loop, you usually want to see the updates from outside the loop.
Eh? Benchmarking on the JVM is notoriously difficult bordering on impossible. There's things like Google Caliper but test runs take forever due to attempting to force JIT warmup and doing GCs after every run. And the project's own wiki tells you the results are basically meaningless for a variety of reasons.
Benchmarking things like C++ or Rust are trivial by comparison since when you call method foo() it's gonna do pretty much the same instructions every time. Highly consistent, highly repeatable, highly benchmarkable. Call method foo() in a JVM language and there's not a single person on the planet that can reliably tell you what's going to get executed on the metal.
That's why you typically profile JVM languages rather than benchmarking them.
The JVM is slow in many regimes due to things like JIT warmup. Rather than accepting that many JVM people are hypersensitive: thou must only benchmark in the JVM's best case scenario.
That's completely false. Benchmarking is a core staple of building & evaluating performance-sensitive libraries or other routines. It doesn't work (well) in non-deterministic languages which reduces its usefulness scope, but in things like C++ it's highly useful and reliable for evaluation of libraries and monitoring for regressions.
> The JVM is slow in many regimes due to things like JIT warmup. Rather than accepting that many JVM people are hypersensitive: thou must only benchmark in the JVM's best case scenario.
Well it's not just the JIT that's a problem. It's also things like GC passes. Does that get included in the results or not? Do you force GC passes between runs? How about finalizers? The answers to those questions depends on the state of the rest of the system and the expected workload, it's not something you can just trivially answer or even accommodate in a framework since most of the behavior is up to the particular implementation, which can then further vary based off of command line flags.
Not that this isn't super valuable feedback, but the title is misleading.
One thing that I think does need improvement is the error handling. Far too many idiomatic examples just panic for error handling. There's no good way to just group errors by a higher-level classification and fall through to a generic error handler. As a result, the choice is between panic on error, or very detailed and verbose error handling. There's no middle ground.
Not at all. This is what the ? operator is for (was try! before).
The idiomatic rust way is to have your own Error type that implements From for other Error types that you may encounter (io::Error, Utf8Error, etc.).
Then you can simply just write let result = dosomething()?; for any operation that may fail. If an error happens, function will return with the error.
I don't think you can do it any cleaner, given the constraints Rust is operating under.
Most of the replies pointed out the new ? operator, which I'll have to check out. (One of the things that I like about the Rust community is that they constantly improve.)
The thing is, if you've only handled errors via return codes in C, then Rust's system is a major improvement. The challenge comes once you've programmed with exceptions that have inheritance. It's pretty easy to set filters higher up in the stack and "not care" lower in the stack. This allows you to effectively ignore error handling for functions that only fail in obscure corner cases, because a filter higher up in the stack can handle so many different errors. (Operations that require cleanup can also be written in a way that cleanup code always runs, as some languages support try-finally.)
That's probably the hardest thing for me to adjust to in Rust. Lifetimes take care of the try-finally part, but I still can't figure out how to have generic error filters higher in my stack.
There's a couple of annoyances to it. The documentation is really sparse for something as integral as it aims to be. And there's a frustratingly large number of third-party crates that don't use standard error types and prevent you from calling chain_error to convert from their error types to yours.
But once you get the gist of it, a few lines of boilerplate gives you the ability to handle errors concisely and correctly throughout your program.
If you're designing your own errors then there's some boilerplate you need to allow for the error wrapping, though you can always take the cheap way out and use Box<Error> as your error type (as any error can be wrapped in that).
And there's an RFC that's been accepted (https://github.com/rust-lang/rfcs/blob/master/text/1937-ques...) to allow main to return a Result, so you can use the ? operator in main, which will let you avoid even more unwraps, especially in example code.
>While overall built-in testing capabilities in Rust are good (file it under good things too), the fact that benchmarking is available only for limbo nightly Rust is annoying;
Okay, but what are you comparing it against? Neither C or C++ have builtin benchmarking or even tests.
>If you care about systems programming and safety you’d have at least one or two functions to convert type into a smaller one (e.g. i16/u16 -> u8) and/or check whether the result fits.
I don't understand this one at all. What's wrong with "as"? And if you need a checked conversion just write a small wrapper function? Again, if you're comparing to C or C++ I'd argue that Rust's semantic are friendlier and a lot less error prone at the cost of increased verbosity. C will implicitly cast and promote integer types without asking any questions, Rust makes it explicit.
I understand C promotion rules and I've still managed to get it wrong on occasions, to the point where I now force myself to make all casts explicit in C like I would in Rust. At least this way the intent is obvious, as I think it should be.
>Also the tuple assignments. I’d like to be able to assign multiple variables from a tuple but it’s not possible now. And maybe it would be nice to be able to declare several variables with one let;
So unless I'm missing something something like:
let (a, _, b) = (1, 2, 3)
Yeah, that's always been a minor annoyance of mine too. I intuitively think that
a(b());
let tmp = b();
a(tmp);
I'm guessing that there must be a good reason why it isn't so however.
Note that in C and C++ parameter evaluation order is unspecified so calling functions with potential side effects in parameter lists should be done very carefully. The compiler won't ever stop you though, don't worry. I'm sure the foot will grow back eventually.
>Borrow checker and arrays. [...]
Now this whole section feels a lot like "I'm trying to code in Rust like in C and it doesn't work and it frustrates me". Which is fair, messing around with arrays in C is a lot easier than Rust, there's no doubt about that. C also makes it massively easier to spectacularly shoot yourself in the foot if you mess up.
Yes, if you need uninitialized arrays and these sorts of things you need to use unsafe. But that's mostly optimization and you probably don't want to start implementing your codec with that type of code. After all maybe the compiler will be clever enough to see what you're doing with the buffer and not actually run the init code. And if it doesn't you're free to add the unsafe code later, once your codec works, you have good tests and it's time to optimize more aggressively.
>And that’s why C is still the best language for systems programming—it still lets you to do what you mean (the problem is that most programmers don’t really know what they mean)
Ah, so after the Sufficiently Smart Compiler we have the Sufficiently Smart Coder. That seems like quite a definitive claim for somebody who didn't know about "split_at_mut" in Rust a few lines earlier. Maybe the author should experiment a bit more with the language before making bold statements like these?
It's not rare to find security vulnerabilities in codecs and they tend to be extremely exposed. Maybe it's worth the hassle of making array manipulation slightly less convenient for the sake of security?
>type keyword. Since it’s a keyword it can’t be used as a variable name and many objects have type, you know.
All languages have reserved keywords, that's strictly in bikeshed territory. Of course when you get to a new language with a new set of reserved keyword you have to learn new habits. You never name anything "struct", "class", "switch" or "break" in C or C++ because you're used to have these keywords removed. Rust lets me name variables "class" but not "type". Oh well.
>Not being able to combine if let with some other condition (those nested conditions tend to accumulate rather fast)
I tend to agree with this one, although I guess it could become messy quickly if you mix refutable lets with regular boolean conditions.
That it silently overflows.
> And if you need a checked conversion just write a small wrapper function?
A better suggestion would be to wait for TryFrom/TryInto which IIRC should be stabilised soon-ish.
> So unless I'm missing something something like:
Assignment (https://doc.rust-lang.org/reference/expressions.html#assignm...) not declaration (https://doc.rust-lang.org/reference/statements.html#declarat...). So
(a, b) = (1, 2)
let (mut a, mut b) = (0, 0);
for it in thing {
(a, b) = <something>
}
// use a and b here
Ditto with the tuple thing, C doesn't have tuples and won't let you destructure anything.
I guess my overall point is that these complaints seems to pit Rust against some other high level languages like Common Lisp or Haskell but his conclusion is basically "C rulez, Rust droolz".
A media codec is a very well understood, highly optimized thing with established idioms that are already well-matched to the capabilities of the hardware (and in many cases are taking advantage of hardware features explicitly designed for the codec being implemented!). Demanding that we start writing codecs differently isn't the same thing as telling a app UI developer they need to get used to traits instead of inheritance.
If rust can't handle the existing idioms for this problem area, that may need to be seen as a problem with rust, not the programmer.
I'm sure whatever the original code is doing could be copied directly, including calling raw assembly code (which optimized codecs often end up doing to micro-optimize as much as possible). You'll just need copious amounts of unsafe code.
But then what's the point? You might as well keep your existing code. I guess it would make sense if you wanted to standardize your cobebase and move everything to Rust, but I doubt that's much of a use case nowadays.
Now if you write a brand new codec it might be interesting to start in safe Rust and once you're ready incrementally add unsafe code where it matters to improve performance while limiting the unsafe code as much as possible. Rust gives you the choice, C is just unsafe by default.
Coders have to write things differently if they want a different result, as should be expected. If the only metric is performance at all costs then assembly beats everything else.
Hold on. We aren't talking about modifying the codec itself, or how it's implemented at the machine level. If it weren't straightforward to generate machine code that matches how the codec is designed to be implemented, that would be an issue with Rust—but that isn't what this thread is about. We're only talking about the particular language idioms used to create that implementation.
If you use CMake, you do get tests for free at least.
> Maybe it's worth the hassle of making array manipulation slightly less convenient for the sake of security?
On my computer I will always choose speed over security... especially for video processing & stuff like this. Wasting CPU cycles has a direct effect on my energy bill. Other people may do other tradeoffs.
CMake is not part of C. Tests and (soon) benchmarks are parts of Rust. That makes a big difference in practice.
Besides if third party solutions are considered it won't be difficult to come up with one for Rust.
>On my computer I will always choose speed over security... especially for video processing & stuff like this.
It's true that the prospect of running unoptimized codecs is not very appealing. That being said when I think about it video and audio streams are probably the main source of untrusted 3rd party data that I actually handle on my computer. And it's not simple processing either, it's quite complex.
Imagine the havok if a zeroday in a popular codec library is found and an attacker manages to ship it on a popular tracker in the form of "Game of Thrones S07E04.mkv". The payoff would be immense. I wouldn't run a random executable found on bittorent but I won't think twice about opening a video file.
What's the worst that could happen? It's not like video, image and audio codec implementations get exploited much, right?
In case that sarcasm was too subtle, I think you would be hard pressed to find a popular implementation of common media format (h.264, mp3, JPEG, GIF, etc) that hasn't had an exploit at some time.
It's your choice what to run, but I can't help but be somewhat annoyed at the professed position because it doesn't just affect you when it goes wrong.
> On my computer I will always choose speed over security... especially for video processing & stuff like this. Wasting CPU cycles has a direct effect on my energy bill. Other people may do other tradeoffs.
Rust allows you to make that tradeoff, ridiculous though it may be.
Who's fault is that?
Rust seems like an attempt to take a lot of foreign concepts to C/C++ programmers and dress them up in ALGOLy syntax. If you look at Rust's influences page[1], there's stuff from ML, Haskell, Erlang... and yet Rust code examples I've seen all look like "slightly weird C++." If Rust wasn't meant to be written like C/C++, it could spend less time aping it and more time looking like languages where you expect to come across ADTs and the like.
Sure, but what part of serving them involves dragging them on Hacker News when they try and write C-like code in Rust?
This is true, but it is unfortunate that a relatively common word like "type" has been reserved (whereas you very rarely want to call a variable "struct", and "class" at least has the homophone "klass").
This is actually something that I really like about Swift and miss when I come back to Rust. In Swift, most keywords are contextual keywords, so you can still use them as variables or functions. And for the ones that aren't (or the ones where the usage you want is in the keyword context), there's a syntax using `backticks` that lets you turn it into an identifier anyway (which is useful if the declaration runs afoul of the keyword, but the common usage doesn't, so you can use backticks when declaring and leave them off when using).
That's because lifetimes in Rust are currently tied to a lexical scope - a borrow of a variable is kept alive until the variable goes out of scope, even if it's never used after a certain point in the function.
There's currently an open RFC that would allow the kind of nested calls described in the blog post: https://github.com/rust-lang/rfcs/pull/2025