The 3D Software Rendering Technology of 1998’s Thief: The Dark Project (2011) | Hacker News Reader

The 3D Software Rendering Technology of 1998’s Thief: The Dark Project (2011)

nothings.org

113 pointsluu45 comments

Funnily enough, I've been playing Thief recently (this one, not the rubbish new one). And wow that game is hard.

It's also an absolute blast to play, and an important piece of gaming history (one of the first First Person Stealth games, helped codify 3D stealth as a genre, engine later used in System Shock 2, had some influence on Warren Spector's next game, Deus Ex).

While the renderer wasn't particularly impressive for the time, the AI, sound, and lighting engines were, which was essential for a stealth game.

> And wow that game is hard.

It's brutal. I ended up way down at the far end of a level (a sewer I think) only to find that I was missing something I needed to do or have from back near the beginning. Lack of attention gets punished hard. Great game though, you really feel like you earned beating each level.

Finished that and Thief 2 on max objectives on every level. Not because I'm particularly fond of rock hard gameplay, but because you got to appreciate the game to its fullest extent.

Games that up the difficulty by making the mobs bullet sponges, meh.

(With that said, some levels in the first were a real grind to deal with all the guards.)

I really appreciated the fact that one of the ways they ramped up the difficulty was by increasing the required objectives.

There's literally entire areas you don't need to visit if you use the lower difficulties. On the other hand, the extra cash you get lets you buy a lot more stuff... which you're going to need to accomplish all those objectives.

I don't particularly ever wish to revisit The Cathedral. Spent three hours retracing my steps trying to find about 15 gold...

Well, I finished Thief just before I met my now-wife. In fact I think I finished the last few levels while we were just dating. So it's just as well I didn't go for all the objectives or either I'd not have been too busy to meet her, or I'd have stopped before finishing ;)

The real challenge was to finish the game on Expert, where you didn't have to kill anyone. It was a great system that allowed dedicated players to get most out of the stealth mechanics, while still providing an alternative play style for casual players.

There's an even more difficult playstyle called "ghosting" which 'requires' that the player not even alert a guard past a certain level, let alone knock them out.

It's well worth a try if you've never given ghosting a shot, although the game design doesn't always permit it.

For the ultimate challenge, try playing Theif Zatoichi style: entirely blind.

No, I didn't make that up. Someone on the looking glass forums brought it up as a joke. I don't think anyone has done it yet, though.

Given how many games there are blind runs of, though...

I found it not that hard at least on levels with people. Instead of killing, you just blackjacked everyone and hid them.

If I'm remembering correctly, one of the things that made the higher difficulties more difficult was the fact that you were not allowed to kill anyone.

To be fair, something like half of the first game was full of super-natural creatures (Undead, monsters), which you were both allowed to kill, and had enough supplies to, on any difficulty.

Thief 2 was almost entirely composed of human enemies, and even had some missions where you couldn't knock them out - or even be seen by them.

Aside from the no-kill requirement, higher difficulties reduced the amount of items you could find, increased the number of objectives you needed to accomplish, and greatly increased the number of guards you had to deal with.

And the world was so open, you could play as little or as much of a map as you wanted, given you cleared objectives. Rope arrows took you up buildings if you didn't want to crawl through. Plenty of bystanders to scare off playing ghost if you wanted to crawl every inch of their quarters for silver.

The amazing thing here is that Looking Glass studios was known for its well-written stories and innovative gameplay, in addition to its technology. The first two generally come from highly creative people that are not necessarily trained in the minutiae of high-performance technical design. It must be incredibly difficult to assemble a team like that while also recruiting someone with the technical chops to create the engine.

I remember reading some rumors that part of the dev team behind Thief also works on Dishonored. Thief was a fantastic game, so is Dishonored, and the similarities are striking. It would be interesting to know if this fact is true.

Ironically, Dark Shadows has been one of my all-time-faves because of the great atmo. Today I'm sitting next to my 16-year-old watching him playing Dishonored (+ Skyrim and others) on the Paystation.

This would not surprise me at all. One of the main level designers for Dishonored also did City 17 in Half-Life 2. When I first played through it I was like "wow this person really got inspired by HL2". Turns out it was the same guy. Dishonored was honestly a better Thief reboot than the new Thief, but then again the new Thief really made me furious. I think I got < 2 hours in before I just couldn't stand it anymore.

That's true. At the very least, one of the head designers was on the Thief team.

For those wondering what it looked like: https://youtu.be/38tJGachSfg?t=10m13s

It looks pretty amazing to me and fluid for a purely software rendered video game.

I've seen screenshots of Thief before but I never thought that the actual game is this smooth!

...Actually, I'm pretty sure Thief did get hardware rendering at some point. For one thing, the finished product uses D3D.

You are correct the shipped game absolutely used hardware 3D acceleration .. by 1998 3D cards were quite popular, that was the early nvidia and late 3dfx era.

This was even mentioned in the post.... The author left the company for a short while and returned to add hardware rendering.

In that video, the game is using hardware accelerated 3D because we can clearly see the bilinear-filtering.

In that video it's running on a modern system using hardware accelerated rendering. It didn't look like that back in the day.

This one looks more like Software:

https://www.youtube.com/watch?v=lXLNk99JMfY

because its not software rendered, you "can tell by the pixels"

This game was beautiful and truly exciting to play when it came out. The graphics really contributed to the feeling of being there, sneaking around, watching light dancing from torches and shadows of moving NPCs.

The audio was done well too, gave a real sense of atmosphere. I find their latest installment with all the fancy graphics, high quality models, cutscenes and so on just not as fun to play. They just don't capture the same atmosphere.

The mental abstraction layer primitive graphics provides may communicate more value than we gain from a near accurate simulation.

I'm not sure if this is because realism in a game contains mostly unnecessary information and our brains don't have to filter all of that out, or it is something else. The part of me that is bearish on VR is for this reason.

VR graphics don't have to attempt photo realism. Realism is a particular artistic choice, the technology doesn't force it on you.

It's a problem with a lot of newer games. As someone who remembers playing on the old Atari 2600, the hyper realism gets tiring really fast.

It manifests itself, not just in the graphics, but in forcing me to watch yet another ladder climbing animation, my character not being able to stop on a dime and turn completely around, ad nauseum.

I vastly prefer games that don't go for realism.

The latest installment is something else entirely with the Thief name slapped on it. It's not even the same studio.

Like so many elements, audio is under-appreciated. Indeed, the Thief audio was excellent. Without it, and if it were less professional, the game would have been far less than it was.

I find software 3D rendering really interesting, as somebody who works on indie games. GPU-accelerated rendering is obviously much faster and you can get much better results; but it does have drawbacks!

With GPU rendering, I spend the majority of my time testing and debugging on various platforms. 3D APIs are very complex, hardware capabilities vary a lot and driver quality is very poor. So even though you can get much better results in principle, in practice you can't get it as good as you hoped (or at least I can't) while also making it robust and reliable.

With software rendering, you needed a ton of effort (by really smart programmers like Barrett) to get viable results, but at least you had confidence that it would work on everybody's computer, rather than crashing randomly on drivers that don't implement a 300-page 3D spec correctly.

I'm told that Vulkan should improve matters, being simpler and better-specified than OpenGL, but I'm skeptical. It still seems very complex, and it doesn't run everywhere yet so it's not a great fit for low-end indie games.

"Much Faster" is a bit of an understatement. The last software rendered title I can think of is the first Unreal Tournament. No CPU can handle the load of a modern game's graphics, not even close.

Even badly integrated OpenGL is still orders of magnitude faster than the most optimized software rendering. And you're getting a lot more in terms of image quality on the GPU: texture filtering, anti aliasing, "free" conversions to gamma/linear space and more.

Vulkan IS simpler than OpenGL, theres no arguing about that. But simpler does not mean easier. There's just much less stuff to worry about as engine developers to hit the fast path. On the other hand, you're managing a lot more memory by yourself.

And even in software rendering you still get different results on different hardware. And even different results in debug vs release. Floating point arithmetic is fun like that. CPUs varies in power just like GPUs do; cache sizes vary greatly, number of cores, base clock speed, amount of available memory, memory speed, etc. And we're not even touching SIMD yet.

You can't replace having to learn the hardware and the APIs to access it in order to build a game engine. That was also true of software renderers; they're closely tied to the properties of the CPU they target.

That's why you get access to Unreal4 or Unity3D; very, very few people can actually make stable and portable game engines. Its just a hard problem space to begin with. Making it easier only means losing lots of flexibility and performance.

>The last software rendered title I can think of is the first Unreal Tournament.

Deus Ex had software rendering as well. But that was on an engine derived from the UT one.

"Much Faster" is a bit of an understatement.

Sure, "vastly faster" then, if you like. :)

Vulkan IS simpler than OpenGL

I hope so! I haven't yet used it seriously. I don't care so much about simplicity per se, I just really really hope it's more reliable. I'm skeptical because the design rationale seems to be "make it simple so that implementations can be fast" rather than "make it simple so that implementations are more likely to be correct". Hopefully those will go hand in hand though.

And even in software rendering you still get different results on different hardware

That's true -- I considered editing my post to mention that but felt it wasn't relevant enough. It's true that any high-performance software engine will have tuning for cache efficiency, inline assembler (or at least intrinsics) for SIMD and the like. But I think the situation for software is still easier than for accelerated 3D. 98% of software will Just Work anywhere; the remaining 2% is stuff you've explicitly hand-optimized, and might be fiddly to port, but you can at least write tests for those parts. In accelerated 3D, almost anything can and will fail on some platform, and automated testing is much harder.

You're right that floating-point gets fiddly (but not nearly as much as floating-point on GPUs, which vary enormously in precision!)

And multithreaded software is definitely a different beast. That's very hard, and different CPUs have different memory models. But even there, you aren't typically hampered by bad drivers.

You can't replace having to learn the hardware and the APIs to access it in order to build a game engine.

I think I only partly agree. My beef with 3D hardware acceleration is that while the hardware is ridiculously fast, it's also very flaky, with tremendous differences between platforms even when they claim to implement the same API.

Using a framework like Unreal or Unity helps a lot, but still doesn't entirely fix the problem. Some of Unity's basic demos don't work correctly on all versions of Android (they use textures with alpha, but Android defaults to ETC1 textures which don't support alpha).

Overall, I'm not saying there's a solution to any of this, just lamenting the fact that 3D hardware acceleration is so painful! And I still think it's painful in a qualitatively different way from plain unaccelerated software. Current APIs are simply not dependable. I do hope Vulkan will be a major improvement.

I'm not saying GPU vendors are perfect either - most of them are terrible. I have nothing but curses when it comes to Adrenos GPUs for one. Most of the problems you list come from the vendors, not the APIs in between - it would be a hundred times worse without GL or DX.

Its also worth noting that the GL/DX you see is vastly different than the raw API exposed by the hardware. You don't want to generate command buffers for 200 models of GPU. You don't want to assemble shaders in 200 variants of similar instruction sets.

I'm no hardware guy but I'm sure standardizing the GPU's instruction sets like X86 is would come with its own bag of drawbacks. There's also a lot of historic baggage and hiccups to account for.

I think we're mostly agreeing, and where we're not I bow to what sounds like greater experience. :)

I definitely agree that a) GL and DX help, and b) both are still really painful to work with.

Do you think Vulcan is going to be a big improvement? I'm specifically interested in how confident we can be that our code will run properly more or less everywhere, without lots of testing and debugging on lots of different devices.

Vulkan won't help with correctness. A bad driver or a bad GPU will keep on doing the wrong thing no matter what API you use to fill its command buffer.

I never believed in the "write once, run everywhere" thing to begin with. Its not true with C#/Java, definitely not true with web dev, and most certainly not true with GPUs. It looks good in theory but rarely ever works in practice.

What Vulkan/DX12 helps with is exposing a lot more of the GPU's low-level details to engine programmers. This has the negative impact that all the complexity is now in the hands of engine devs. But the upside is that now finally the engine and the driver can stop guessing about what each other is doing.

That was a good thing decades ago when APIs were much, much simpler than they are today. Nowadays its near impossible to hit the fast path of GL4/DX11 without serious investment because of everything the driver does to "protect" you and make your life "easier"; it grew into a complexity monster.

So its actually harder to use Vulkan than it is to use OpenGL. But once you learn it it becomes simpler to work with.

One thing to consider there is that today most 2D inide games are in reality rendered by 3D hardware pipeline (because of some middleware that just maps anything to Direct3D/OpenGL ES, today this includes even SDL) and that is the use case where you (a) don't need hardware acceleration for rendering at all and (b) trigger obscure driver/hardware bugs that nobody cares about because they are irrelevant for truly 3D engines.

So long as your 2D game doesn't have lighting or require shaders. You'd limit yourself to vector rasterizing and blitting bitmaps.

There are loads of 2D games out there pushing the GPU far enough that they would never run in software rendering.

For some background, I've been writing 3D code since the early '90's and spent years working at SGI, games studios, all doing realtime 3D.

The tech pendulum swings between solving problems in hardware vs software constantly. Right now, we're somewhere between the two endpoints with the way shaders moved a lot of ASIC behavior into software.

Software vs hardware 3D rendering speed is just one aspect of the differences between both approaches. Hardware graphics pipelines all share a common design - a high throughput deep pipeline which maximizes geometry and pixel throughput. The hardware is incredibly powerful, but you're still communicating with it via a thin little unidirectional pipe relative to its performance, so by using hardware, you end up having a common architecture in rendering engines; that of a retain mode renderer which tries to upload as much static content to the card as possible, then spends whatever time is possible on animated content. That's why today's games all look positively glorious and beautifully shaded, but if you look close, they're all fundamentally a few characters moving around a large static world. The 3D hardware systems simply aren't optimized for dynamically changing all that much of the scene.

Back when we did pure software rendering, we had much more flexibility in our approaches, and there was a lot more dynamic rendering - fully deformable worlds. I miss the huge variety of games which were possible back then. They are still possible today, since hardware is far more capable, but due to being off the optimized path, you can't compete on look and feel, so you lose in the market.

A small incise : On 2001 Blade, the edge of darkness (aka Severance : Blade of darkness outside of Spain), did a really nice job displaying indoor/outdoor dungeon ambient.

The ambient, and the water effect and realtime shadows was really awesome.

Reply on news.ycombinator.com