A single character code change

Here's what he changed:

> Set<Thing> aSet = new HashSet<Thing>();

to:

> Set<Thing> aSet = new HashSet<Thing>(0);

His explanation:

"Most of these sets were empty for the entirety of their life, but each one was consuming enough memory to hold the default number of entries. Huge numbers of these empty sets were created and together they consumed over half a gigabyte of memory. Over a third of our available heap."

I'm not sure he ever got around to explaining how it saved "half a million dollars", though.

In this story, instead of doing meetings, why did no one fire up the profiler and actually try to figure out why the heap size was so big? If the senior architect's first thought is to come up with an elaborate proposal even before they understand the memory and processing budgets of their program, what are they a senior architect for? It's like trying to shave off micro seconds of processing time while your network call takes milliseconds.

Good style guides strike the balance where common caveats and easy to miss gotchas are avoided by making rules around them (and then it's no longer premature optimization because someone already did the math and decided on the optimization, so each new programmer using it is not optimizing but merely following a rule in the style guide), while leaving the complex cases for the programmer to optimize as and when needed.

Many people who scoff at opinionated style guides forget that in a big company you don't want 1000 engineers to spend time independently figuring out whether X needs to be optimized [1]. If someone can do the profiling, figure out what should be done in 99% of cases, make a style rule out of it explaining the rationale, it saves everyone the hassle. I believe Google does exactly this.

[1] Apart from the waste of time, not every engineer might be capable of actually figuring it out. Many of the intricacies of C++ for example aren't immediately obvious even if you have spent years with the language. It's more optimal to hire a few C++ super-experts, have them do the profiling, and come up with style guidance.

I had sort of the opposite with a Java 1.2 code base. Some clever person has noticed that a bunch of vectors often had only 4-7 entries so they overrode the default (10) to 7. This is unfortunately greater than n/2 which will become important momentarily.

In Java collections tend to double in size on space exhaustion. Using a multiple instead of addition amortizes the insertion time to O(1). So if you start at 7, you go to 14 next.

As I’m sure you can guess, the number of entries crept up to 8 over time. Switching back to the defaults reduced the wasted space from 6 to 2 slots, and dropped our memory usage by more than 10%.

V8 allocates a whopping 64 bytes per empty object.

    {}

    {v8Hack: null}

Unfortunately, there is no size hint like for Java's HashMap; the closest hint you can do is create your objects with dummy properties.

[1] https://stackoverflow.com/questions/59044239/why-do-empty-ob...

[2] https://www.mattzeunert.com/2017/03/29/v8-object-size.html

In particular, you should always aim to profile your app when running under a production load so that you do not have to make assumptions about its behaviour. Something like async-profiler[0] is good, since it avoids the safepoint bias issue and can also track heap allocations.

0. https://github.com/jvm-profiling-tools/async-profiler

So it really is possible.

As a DBA, I routinely save that (and more) by doing query performance tuning. I aim for 500x-1,000x faster on physical hardware, and 10,000x in the cloud, thanks to EBS/pSSD latency.

And not using Hadoop.

But even worse, the "fix" was what I would consider a spaghetti hack. They create a HashSet that is mostly never used? The real fix is to create the HashSet when you know you need it and don't create it if you don't need it. It sounds like the underlying code has expectations that the HashSet exists and is valid, which in and of itself is bad code.

Check to see if it's null, and if it is null and you need it to not be null, then allocate it then, with size 0 or default capacity, whichever makes more sense.

One of the better cases for using appropriate comments...

There are some good points, but, like so many of these topics, it enters the world of "orthodoxy," where we all have to do things the same way, everywhere, and in all places.

Optimization (effective optimization, anyway) is a fairly intense process. For one thing, common sense doesn't really apply. What seems to be an optimization can sometimes do exactly the opposite, like breaking cache, or introducing thread contention.

It can also introduce really weird bugs, that are hard to track down, and strange code structures that relatively inexperienced maintenance programmers may have difficulty grokking.

It's all about the metrics. Profilers are your friend.

Also, in many cases, optimization isn't necessary at all. If the code controls response to a tab selection, then the code that redraws the tab is likely to be executed in a separate thread, anyway, and done at the pleasure of the OS. Why bother optimizing the lookup index to save a few microseconds?

Another matter, entirely, when we are iterating an Array that is many thousands of elements long. In some cases, using HOF can actually decrease the performance (but YMMV). I have sometimes had to replace a map() with a for.

We measure and find hot spots, and then concentrate on them.

If we are working with mobile or embedded, then we also need to worry about power consumption. That's a fairly fraught area, right there, and optimization can actually cause power drain.

That might be some record both for inefficient coding and then for not caring much about inefficient coding and all the extra expense millions of WordPress users face to host such inefficient code? Either that or maybe few WordPress users make many edits to large pages?

As much as developer time is more expensive than machine time it seems people are forgetting best practices and/or minor improvements that can have a big impact.

  The form:
  Boolean b = Boolean.valueOf(true);
  Is slightly more efficient than the subjectively uglier:
  Boolean b = new Boolean(true);

  Boolean a = new Boolean(true);
  Boolean b = new Boolean(true);
  System.out.println(a == b);
  
  false

Leads me to believe that there's lots more meaningful optimizations remaining that someone who better understands the system could implement.

Ah, but the object still exists. If large space of objects is needed that is sparsely used, the thing to do is to make that space itself lazy, not the individual objects.