Dirty Coding Tricks | Hacker News Reader

Dirty Coding Tricks

gamasutra.com

304 pointsLaSombra88 comments

That last one about the guy who tucked away 2 extra mb of memory for a rainy day is one of the most bad ass things I've ever read about a programmer (never thought I'd say those words). That's next level, old vet, been around the block a few times, stuff right there.

When designing performance-sensitive software, database engines in my case, it is not uncommon to "bank" optimizations and resources for future code emergencies.

From a customer's perspective, you are always allowed to make the software faster with a future patch but never slower. The issue is that sometimes you find bugs (or add/complete features) where the correct implementation is somewhat slower than what you already shipped or planned for. In these cases, you end up creating two patches: one to address the customer issue and one to add an optimization to return the overall system performance to its prior level.

I've shipped complex v1.0 software where low-hanging optimizations would probably have made the system 2-5x faster (though still perceived as being extremely fast). Over time, it allowed us to add in expensive features and bug fixes without perceptible reductions in performance by harvesting the optimizations we'd banked in the first version.

I learned something from this. I'm saving this snippet of advice for future reference.

Me too, its a great bit of advice that I had never even considered before! bookmarked!

So Scotty-based software development?

Back in the early 90's when I was in college and everyone had email accounts on shared unix servers that they'd log into, a sysadmin friend would oftentimes notice that the disk would be getting close to full. He'd start a process that would slowly fill up disk space, and wait as all the other logged in users would realize they could no longer save files - then they'd frantically start to delete files they no longer needed. After a while he'd cancel his process and the system would have a lot more space.

The other way around:

In 1995 I worked in a Dutch research institute where the file server was always almost out of disk space. Whenever some space freed up, my manager used to create something he called a 'balloon': a directory with 20 files of 5 MB each (which was huge back then).

If the disk was full and we needed space, he just removed one of those files and we could quickly write our files.

I think Second Life had something similar early on, way before I started playing. It let users build stuff out of basic shapes called prims, and the number of prims in each region was limited for performance reasons. So users started to set up "prim banks" of unnecessary prims they could delete when they wanted to build something. The developers had to add quotas in the end.

I have worked in buildings low on meeting space where middle managers would do something similar, but with meeting rooms. You can always find a meeting if you abuse the system to 'claim' one room as your own for enough hours of the week.

Hoarding in a situation of some scarcity is common behavior, and makes the problem worse. You can't assume people will behave the same way when a resource's availability changes. And this is why we go from plentiful to very scarce so very quickly, with very little in between

tragedy of the commons... or is it comedy?

it is more that people today take for granted huge amount of resources at their disposal

you become VERY creative when hard times hit :-)

You only become creative when faced with limitations.

Is boredom a limitation?

Poor mans superuser reserved space :)

Kind of related to this, I find it interesting that the ext file systems by default reserves a percentage of the space for the root user.

All native Unix filesystems do this (well...at least since V7; I don't remember about V6 or before). It's so the root user (and the kernel, and applications running as root, e.g. logging) would have some amount of disk space to work with if needed. Much more important in the days when typical servers had less than 100MB of disk.

The default is (usually) 5%. In these more bountiful days of multi-terabyte disks I generally knock that down to 1%.

I came across code in a legacy web application one day, on a page that was probably the most used part of the application that was really nothing more than sleep(1000); I went through the version control history and found that is was a previous employee that would go in periodically and lower it any time he wanted to act like he had been working - he had started around 3.5 seconds.

Which reminds me of being a kid and mom telling me to turn down the tv - I would "accidentally" hit the wrong button and turn it way up for a few minutes, then just turn it back down to where it was.

I just recently heard a story that went the other way.

An engineer of elevator systems would spend hours doing all his calculations by hand, vowing it was too complicated and important to trust to a computer. (Yeah.)

The programmers in the company assured him they could write a program to do the calculations for him, which they did. He never trusted the results though, insisting that the program was not producing correct results.

One of the programmers realised that he didn't trust it because the program spat out the results almost instantly. It was too fast, it had to be wrong.

So he stuck a twenty second sleep in before the output, told the engineer he found the problem and fixed it. The engineer now felt the results were correct.

(Aside: I don't think I would want to get in a lift designed by an engineer who intuits the correctness rather than verifying the math, but I thought it was an interesting little parable on user interface design.)

Kayak does this when looking for flights. The results are instant, but by taking more time, it's giving the impression that it's trying really hard.

Relevant discussion on the UX Stack Exchange: http://ux.stackexchange.com/questions/41682/pros-cons-of-spl... (which also links back to a story on HN about a locksmith.)

That sounds a bit like the cake mix people who discovered that they got better sales if the mix required an egg to be broken and beaten into the mix. The taste was no different, but the cooks felt better. Or the antiseptic spray that added a component to give the sting consumers expected.

If I ever found out someone did this not only would I fire them, I wouldn't keep it a secret. I'd make sure everyone on the team knew exactly why he was fired. That is acting is gross bad faith. Think of the time it had cost all those people around the world just because he wanted a day off here and there. Blows my mind that someone would do that.

With my example I agree with you - the example in the article is different though - putting artificial constraints into a project to encourage innovation and making sure you have the necessary resources - and then removing the constraint before you go live - isn't a bad idea IMO, especially when you are dependent on multiple teams. Think of it as resistance training :)

Not to defend this practice, but I heard a story about a whole team who put in a sleep that would automatically shrink over the first 6 months it was in production.

I understand where you're coming from, and I agree with you, but we can't pretend that 1 second from 10,000 people is the even close to the same at 10,000 seconds from 1 person (or really anywhere in between).

The entire concept of "we saved 10 million hours of productivity by shaving 1.8 seconds off this process" is bunk. You saved 1.8 seconds of productivity n million times, which adds up to not a whole lot of extra productivity.

Put another way: Whether $APP takes 0.1 seconds or 1.9 seconds to do something does not at all change what time I'm leaving at the end of the day.

User behavior when interacting with a program that responds in 0.1 seconds is drastically different than with a program that responds in 1.9 seconds; see [0] [1] etc. I agree it's not very meaningful to just compute the total number of seconds saved, but on the other hand, the productivity gain could easily be far greater than what that number suggests too.

[0] http://googleresearch.blogspot.com/2009/06/speed-matters.htm... [1] http://www.webperformancematters.com/journal/2007/7/10/accep...

> Put another way: Whether $APP takes 0.1 seconds or 1.9 seconds to do something does not at all change what time I'm leaving at the end of the day.

But it does greatly change how annoyed you leave at the end of day.

Not to mention any noticeable delay in load times will quickly get people to start multitasking, which can easily change a 2s delay into a 2min delay. This can easily ... well it does make a difference.

I noticed this mostly while testing code. After clicking through the interface many times it can easily add up to an hour less actual work getting done that day.

Sometimes 2 or 3 more second is all it takes for the user to switch to procrastination "while it's loading" and waste much more time.

Companies do this on a much bigger scale though. The logic of planned obsolescence is very much the same. You get more benefits doing worse than what you could do.

This is a pretty common sales & negotiation technique. Anchor high, with a much higher profit margin than you actually need. Do your best to get the customer to buy at the high price. Periodically, when you want to seem benevolent, offer a "sale" or "concession" that gets you down to what you were willing & able to do in the first place.

Every time you buy stuff on sale at the grocery store, you're seeing it in action, but you probably don't think much about it.

I first heard of "The Speed Up Loop" here: http://thedailywtf.com/articles/The-Speedup-Loop

From my experience in the games industry, this technique is fairly common. Not so much in the sense of 'hiding' megs away from other programmers, but having team buffer space (by 'team' I mean engine-team, AI team,...).

For example as an engine programmer you'd want to pre-allocate space for various resources (models, textures, etc.); some stuff is dynamic, like the currently active game entities. So you'd have preallocated chunks of memory for N entities, where N is the maximum number of entities you could handle at that time (at every opportunity you'd want to avoid classic memory allocators because they're slow and a fragmented heap could kill your game on a limited memory device; so everything would be done with fixed block-size, fixed length, linked-lists). It was relatively easy to 'hide' buffer space for a rainy day because the list sizes would just be magic numbers that were decided as 'roughly what we'll need for later'.

They could be per level magic numbers, or system wide magic numbers. Nobody gives a shit about the magic numbers at the start of the project on development devices with more memory than the final device, and mostly nobody even knows what the game's going to be early on (this was something I saw a lot in the games industry: teams just feeling in the dark for what it is they're supposed to be doing for months on end).

Towards the end of a project when things are tight, and you have a better sense of how much of each type of thing you'll need, that's when you start giving up the buffer.

Reading TFA reminded me of how much I do not miss working in the games industry. It was fun while it lasted (I did it for 10 years, I left 10 years ago), but after a while this style of programming drags you down. It's too much stress, for mostly not enough reward.

The last game I worked on: https://www.youtube.com/watch?v=G9zMWu6PfSM

When writing safety critical software for limited embedded hardware we had a policy that new features were strictly not allowed to use up the final x kilobytes of the memory, these were reserved for critical safety patches. You don't want to hardware recall millions of units because memory is too low for a software update. This was handled by a build step though, not someone secretly hiding allocation in the actual code.

I've read that story before and always wondered what kind of profiling they were doing that apparently allowed those two megabytes to "fly under the radar".

The CPU equivalent would be the "speed up loop" that somehow avoids being recognized by a profiler.

I wonder if there are more 'buffers' in the program if further optimizations are needed

he was toasted as the hero of the project.

Sorry, but this guy sounds like an asshole. Derailing a project so you can be the one to save it is pretty sleazy. What about the stress on his colleagues, the late nights, etc? They were "frantic" and "exhausted", and he was sniggering the whole time. The human cost of his moment of glory?

They still only came under budget by half a MB. That stress they put into reducing it was very likely still entirely justified. It's like setting your alarm clock 10 minutes fast to hurry you out the door in the morning.

Taking credit for the last 2MB is a dick move for sure though.

But is it really a jerk move? He had the foresight to tie up those 2MB at the very start so that later on it could be removed. And in the scenario written about, doing that ahead of time saved the project and major content from being cut.

Yeah it's a jerk move.

Think what would have happened if he hadn't added the alloc. The team would have been in the same situation, would have busted their guts, got it down under budget and slapped themselves on the back. They might have got it down earlier too: who knows how many person hours got wasted on the last 384K of savings that were never needed.

Think what could have happened with the alloc: the team is hovering around 3-4MB above budget and running out of ideas, with major efforts shedding only a few tens of KB each. They agree they might manage to cut 1-2Mb with a herculean effort, but 4? No way. So with the deadlines approaching decisions are made to cut some feature or content to avoid failing TRCs or missing contracted delivery dates.

If you have to get it down under target or cut something else, then I can't think of a single situation in which having an artificially low goal will help you.

At the very least, the fact that the team thought this guy somehow helped them ship rather than adding to their problems and lauding it over the actual engineers who'd bust their ass to make the savings, is a huge jerk move.

"He called me into his office, and we set out upon what I imagined would be another exhausting session of freeing up memory."

This itself implies that the guy WAS one of the "actual engineers" busting their asses.

The only thing is, the content team was building to a budget from the start. So they would have, for example, just planned to use smaller textures if the early memory budget pinched them.

Now, whether or not you actively hide a budget padding is another thing. That is up to the seniors of the project.

It's basically parkinson's law[1].

As a project manager you are also supposed to pad your project schedule a bit without showing your client or engineers the padding.

[1]http://en.wikipedia.org/wiki/Parkinson%27s_law

Yes, it is.

I knew some people who wrote a graphical demo at a small, local demoscene party. One of their effects had a static, vertical tearing artifact that moved around based on their parameters but which they weren't able to entirely get rid of.

One of them noticed that the projector screen that the demos would be shown on had a vertical splice. He eyeballed the location and tweaked the parameters to make the artifact line up with it.

No one noticed.

My assembly classes at undergrad had a final project consisting of writting a tetris clone in x86 assembly. Shortly after the deadline, my program had a bug that made pieces fall up, instead of down, and I was completely unable to find it's cause...

In the end, I just rewrote the score checker to verify the top of the game, instead of the bottom, and delivered a game where pieces felt down. Everybody not just noticed, but I had a line of people wanting to play it.

I'm Mark Cooke, I wrote the first entry in this article a few years ago. Of all of them, it's the least related to interesting code, but it does reflect the reality of the kinds of crazy last minute hacks that can go into shipping a game.

Unlike many other types of reusable code, this sort of gameplay hack will be thrown away and won't be used again in a subsequent game. To an extent it justifies its use in a pinch. But no craftsman is ever happy shipping something like it.

It has been my experience that at the end of game development projects many of these sorts of hacks start flying. Most of them are unnoticed by players.

No game is ever perfect. With limited resources and time it's important to keep in mind what players care about and focus on those things. I care deeply about providing a good experience to players and one of the most interesting and challenging components of game development is deciding where to apply resources when under tight constraints to deliver the best experience. I haven't worked outside the game industry in a long time but I'm sure this is true in most consumer software businesses.

Out of curiosity, what was the game you were working on in the article?

One of my favorite tricks is this one (I thought I read about it in that exact collection but it isn't there, so here you go):

Back on Wing Commander 1 we were getting an exception from our EMM386 memory manager when we exited the game. We'd clear the screen and a single line would print out, something like "EMM386 Memory manager error. Blah blah blah." We had to ship ASAP. So I hex edited the error in the memory manager itself to read "Thank you for playing Wing Commander."

My choice would have been "640K ought to be enough for anybody".

(This will be understood only by those who lost countless hours tweaking their autoexec.bat and config.sys to squeeze that last KB of memory needed to run the game they were supposed to spend the weekend playing ).

Yes!! I love that!

I always try to explain to people I work with that you can't get away with hacks at the deadline if you've been hacking from the start, but if 90% of your system is an ivory tower, it usually doesn't hurt that 10% is a nefarious back alley.

That packing the controller id into a pointer, which you couldn't use to point at a controller id because it gets freed before you can use it?

I must misunderstand something because the hack is going to fail too, except that you are going to end up with some controller id in your allocator's free list or some other such nonsense...

That last one is best. After having to struggle and fight for 640k for /the heart of a renderer/ if someone had left 2MB lying around that would have been great. Then again so would not using loads of memory for all sorts of weird and wonderful things that are much less important... like C struct padding, or making everything into memory pools, before you have any memory problems. :)

You'd probably only need two or three bits to pack a controller id. Depending on how much memory the system has, and and things like data alignment, you can find some bits in the pointer that will always have a specific value (say, zero). Then, you can store the ID in those bits, and mask them away when you actually want the pointer.

Edit: You probably shouldn't do this in new code, but in the context of memory-constrained C code it's a nice hack.

Interesting enough, the current 64-bit ObjC runtime does something similar, with so-called "tagged pointers".

https://www.mikeash.com/pyblog/friday-qa-2013-09-27-arm64-an...

right, but then why can't you take that same opportunity to pull your pointer out and replace it with a zero to avoid it being free'd ?

re: controller id into a pointer

If I understood correctly, the pointer is just a field in a common struct (common in event-handling code). As he mentions, it's usually used to point to extra data that you allocated separately in case you need it for that specific type of event, but what if you're having trouble with allocation and your data conveniently fits in the pointer field itself? So he just cast his controller id into the unused pointer parameter. He's not mangling any allocator data.

Which, if you're not going to use that pointer parameter otherwise, I think is perfectly sensible (and as long as you don't have code along the line of if (ptr != NULL) free(ptr);). I do it all the time... cough. For example, when creating a thread, you can pass it a void* to some data for it. I just cast a threadID integer into there. Hell, you'll see just that in the first example on a google search for "pthread example"!

Apparently there was top level code (that couldn't be changed as it was needed several other places) that DID free the pointer though...

hopefully the controller id was just the lowest bit or two, which would then align back to zero or something when it came time to free ;-)

yeah, so he said the problem was that this pointer was freed out of his control. any way of packing data into that pointer suffers the same problem - except even worse, you are now freeing something that wasn't allocated.

The pointer would get freed later, but freeing a pointer does not change the pointer address. Eg: If the pointer is storing the address "10" and you free 10, the value 10 is not lost, the value 10 points to gets freed though.

Assuming the controller ID would normally point to somewhere in protected memory, the frees would have no effect, and he could store the controller ID in the pointer parameter as the address of the pointer.

But the problem would be attempting to free the pointer which points to some random memory location (the address "10") - I'd expect it to end with a segfault with a high probability.

i've not yet worked on a game console platform that doesn't end up in trouble when you try and free some small number. given the context of doom 3 engine i think my experience covers it well... unless they did some insane PS1 port for reasons unknown.

maybe they used a custom allocator though... its very common.

Great article. I'm not a game developer but I enjoyed reading about the constraints they face daily and the solutions they used and how those eventually backfired. I've always figured I'd never want to be a game developer but this gave the feeling I might enjoy it a bit more than I think. Boy, the experience of seeing a fully 3D game run and and run well, that must feel pretty good.

I really liked the final problem's solution!

Anyways, just rambling a little :-).

I totally agree with you. For me, this is what programming is all about - creative problem solving :).

  Spurred on by the success of "if A==bad then NOT A", I used this tool to patch several more bugs -- which nearly all had to do with the collision code.

This reminds me so much of a developer I work with, why figure out the root cause when you can throw a try / except / pass around it. If that doesn't work upgrade a package and see if that helps. If that doesn't work throw in a hacky fix.

OMG what about those

They're also the first to blame "GCC/Python/JVM/etc" and saying there's a bug there.

Seen this a few times before, but I am always entertained on reread and enjoy the reminder that sometimes you need a scalpel and other times a sledgehammer.

Always is a great read. Can't help myself into rereading these stories every time I see this post.

This is a good reminder for the "avoid technical debt" crowd and other advocates of sterilised development practices

Sometimes the rubber hits the road and you don't have time or money or any more cognitive energy to understand why your code that calculates a lot of things from a lot of data is not working in that 0.01% of cases

So you come up with a quick fix that's not exactly what it has to be done but it works it ships and it pays the bills.

Of course I don't advocate such solutions on critical paths, ate least not without some double checking.

You do want to have your default setting be "avoid technical debt", so that these sort of hacks are just that - hacks. Projects get into trouble when people do them as a routine matter; they're saved when a project that's otherwise healthy needs to do something naughty to work around a tricky bug.

You're right, however, there are two issues.

1 - People "avoiding technical debt" by overengineering.

2 - (As you mentioned) sometimes it's not even your fault that you need a hack. (It may not even be a bug on what you're using, but just an unfortunate impedance mismatch, like the Playstation problem mentioned)

Games are something of a different beast, at least at the time the article was written, in that they have a fixed 'end' point. After the game was shipped you didn't go back to fix things later, things would only be fixed when you were starting the sequel (or some other game) with that engine.

After all, what gets burned on a PS1 CD or PS2 DVD is permanent.

I'm sure that's at least somewhat different today with so many games having scheduled DLC and other updates that make the technical debt more unpalatable, although I'm sure such things still go on at a high rate.

Ocarina of Time, and probably other games, had some glitches fixed, but sure, you have to ship your first client something playable beginning to end, because this client's media is permanent

Also worth reading is the three-part article by Patrick Wyatt on the issues encountered whilst developing Starcraft[1], which were a huge factor in its delayed release. Of particular interest is the path-finding hack they used [2]

[1] http://www.codeofhonor.com/blog/tough-times-on-the-road-to-s...

[2] http://www.codeofhonor.com/blog/the-starcraft-path-finding-h...

We once had a game demo that was crashing every time we tried to exit the game. Inserted an exit(0); in the shutdown code and it never crashed again.

+1. That should always work as an exmaple of "crash only" programming. It's nice to get shutdown working for valgrind, but it should never be necessary.

Some of these are dirty, but I think the majority are smart hacks in the best sense of the word, especially the final story about memory allocation

These seem to come from here [0] which has more and some other general tips.

[0] http://www.dodgycoder.net/2012/02/coding-tricks-of-game-deve...

It's the other way around, the gamasutra article is from 2009

(2009)

Yeah, I was wondering about that. I had certainly read it before, but was confused about the "Copyright 2015" at the bottom of the page. If you follow the link to the non-print version of the article you get the real date.

Ah ha, I was wondering this too since I've seen it do the rounds before.

The print version is so much more readable with a wider body and no comment clutter!

I find th CRC collision story rather unbelievable.

Most people find this kind of thing unbelievable, which is why they call it a paradox (the "birthday paradox"). Assuming a perfect 64-bit hash, and "tens of thousands of files" meaning 20,000 files, the chance of collision is one in 46 billion.

However, if they reuse filenames, then the chance of collision is much higher. For example, assuming 1000 distinct filenames chosen at random, the chance of collision rises to about 1 in 10,000.

You see, shortly after leaving his boss's office and learning about the hidden-allocated 2MB, that guy implemented his own CRC that would throw the same hash just before release.

Maybe they were using 32-bit integers as their crc?

Apparently with CRC32 you only need about 77,000 items to have a 50% chance of a collision by birthday paradox.

It's not a "true" almost impossible collision that occurred, but two files that shared the same content and filename and would collide by definition.

They shared the filename, but the story says "even though they were completely different", IE, not the same content. It was a hash collision; certainly rare, but enough project with enough files around the world and it's bound to have happened to somebody.

Reply on news.ycombinator.com