Strings Are Evil

Yeah, you could easily argue (as I've had many, many bosses do) that the final version is objectively worse as it's doubtful that it will save as much time as it cost to develop (and debug - I doubt he got it right on the first pass!) and will drive in ongoing maintenance: the first version is simple and easy to understand (but inefficient) while the final version is almost impenetrable unless you read through the whole blog post, which is probably not part of source code control.

Explaining the circumstances behind this particular optimization target would certainly have been more helpful; as a standalone exercise, reducing allocations does look rather useless.

But the author presumably has some reason for focusing on allocations (and GC generations, though that never really comes up). Perhaps the subject code is causing excessive GC pauses in the larger program it is part of? If they're doing "real-time calculations" as part of financial services, I can see where adding latency for several seconds could be a bad thing.

> The timing went from 8.7 seconds to 6.7 seconds. So all that effort bought a total of a 25% improvement in memory usage and CPU time.

AFAIK .NET has a concurrent garbage collector. For all we know there was a 2nd core running 100% to collect strings that is now almost entirely idle. It's possible the effort more than doubled overall throughput if you consider the ability to run multiple jobs.

That said, ad hoc parsing is a fundamentally messy approach in every language, not just languages like C with poor string handling.[1] Parser generators exist for a good reason--easier to read and great performance without even trying. Find a good one for your language of choice. The learning curve is steep but you'll never look back.

[1] Actually, I find C one of the best languages for parsing as you're much less likely to conflate string manipulation with parsing. Though that realization comes more quickly for some than for others.

That was my first thought, I do ETL work at a large insurance company and we use shell tools almost exclusively unless we need to do more complex transformations. Unix tools are really low overhead and work extremely well for this task. Also, you can parallelize a lot of them with xargs: https://adamdrake.com/command-line-tools-can-be-235x-faster-...

Absolutely - use the simplest available approach first.

Yes this is one aspect that surprised me about Python.

I expect a scripting language to be really good with strings and I/O. Python does a pretty good job at both. Strings are immutable, which IMO is a good choice and then I get this really disappointing surprise that slices don't share content. I understand their motivation though.

Memoryviews can substitute for immutable string slices at times, but since these are not strings, its annoying. One can get a string out of a memoryview but I believe Python copies at that point.

> Rust's splitting iterators over `&str` return references to the same data as their own `&str`s, which is a huge performance win.

Seems similar to Swift's Substrings (and other collection "slices"):

https://developer.apple.com/documentation/swift/substring

Zero-copy substring classes are hugely useful for parsing. I first encountered this approach in the Zeus Web Server back in the early 2000s, in non-STL C++.

Heck, even in Java, String.split returns shared references (with different offsets) to the same backing byte array.

Mind you, that can bite you pretty hard if you're taking in giant documents, splitting down to a few tiny tokens, and then holding those tiny tokens in a long-lived in-memory cache (ugh, voice of experience here) but most of the time that's the Right Thing.

It's sometimes amazing what low-hanging fruit can be kicking around that you didn't realize until you actually strap up a profiler to something.

The other day I was trying to figure out why one web request in an application was taking an absurd amount of time. I started looking at SQL queries and other external calls, only to realize that it turned out to be a logging statement that used a little innocent looking helper function to stringify a List<T>. Except it was using concatenation, with Aggregate e.g. Aggregate("", (a, b)=> a + b + ", ")... Replace that with a StringBuilder, and I went from seconds to milliseconds on that call. It's one of those things that you go "Eh, I'm just doing this on a few items, it'll be fine", and then month or years later, the wheels come off because something happens to try using it with tens of thousands of items.

I'm currently in extreme optimization mode in C++, and even assembly & hardware some days. BUT -- the first rule of optimization is to only fix measurable, perceivable problems. This article was fun, but it didn't measure the right thing. The sum of all allocated memory is an almost useless metric, it doesn't reveal a problem to fix, and reducing it doesn't necessarily result in perceivable improvements.

I love Michael Abrash's introduction to the philosophy of optimization in the Black Book of Graphics:

"Before we can create high-performance code, we must understand what high performance is. The objective (not always attained) in creating high-performance software is to make the software able to carry out its appointed tasks so rapidly that it responds instantaneously, as far as the user is concerned. In other words, high-performance code should ideally run so fast that any further improvement in the code would be pointless.

Notice that the above definition most emphatically does not say anything about making the software as fast as possible. It also does not say anything about using assembly language, or an optimizing compiler, or, for that matter, a compiler at all. It also doesn’t say anything about how the code was designed and written. What it does say is that high-performance code shouldn’t get in the user’s way—and that’s all."

http://www.jagregory.com/abrash-black-book/#introduction

My thought too, but he did mention that it was running in off hours due to memory constraints. With these optimizations, maybe they can improve their business process to offer this functionality in realtime for the customer. If not, I agree that the extra complexity and maintainability might not be worth it.

I don't know about the rest but 2 seconds doesn't seem negligible to me. And while renting more servers might solve your throughput problem, you also need to consider latency.

Slow code is a plague. Cloud architectures make it easy to buy more power. And if you ignore small but real performance improvements because the cloud is cheaper than devs, you will end up with bigger and bigger bills, and soon enough, it won't be cheap anymore. Even worse, your tasks may end up being too big. And if you don't think about optimization, the answer is to split the big task into smaller tasks. It goes well with the cloud model, it is scalable, etc... very nice. But now, you are introducing possibly more complexity than old fashioned optimization, and your code new requires even more resources for all that I/O. That's how companies end up paying thousands to cloud providers for something that could fit in a single server.

It isn't, really. Everything has a transform to a string, and in a pinch, everything can be converted to a string, have string operations performed on it, and converted back to the target type transparently, but under the hood, TCL uses numbers and such whenever possible. If you add ten number together, at most, it'll convert ten strings to numbers; it doesn't convert the first two strings to a number, add them, convert that back to a string, the convert the partial total back to a number and the third number to a number to add them, etc. The language implements a string-based interface to everything, but the runtime is not literally dealing in nothing but strings.