Hacker News Reader

One note about Id and their open source: they "only" release the source for their old engines, the most recent one is closed and licensed.

Giving something away -after- it has earned you a billion is less risky than doing it -before- you've earned the money.

Right on. But it's as close as anyone has come to "shipping the source code along with the CD". So even though Id did it after it generated a lot of revenue, they still did it, which is extremely rare.

Also, one reason that Id's most recent engine (Id Tech 5) is closed and licensed is because of its brand-new technology: MegaTexture. As of now, no other engine has implemented that feature (or the engine isn't production-ready like Id Tech 5 is). So yeah, if you invent paradigm-shifting new technology, then it is probably a bad idea to immediately open-source it. :)

I agree, it's incredibly generous of them and very helpful for the next generation of game developers to be able to read and use "real" code like this.

And I think that more or less all their engines have been revolutionary when they were released, not just this one!

Some other options: You could open source the engine, but not code that is specific to the game (then people couldn't run the game). For a legal distinction, you can also make source available, without making it open source. e.g. "look, modify but don't redistribute." See the creative commons licenses. Or make it available only to buyers of the software.

A benefit to open sourcing the engine: it would make it possible to become a platform, with other games built on it - and the publicity etc could help you. (Of course, open-sourcing isn't enough to automatically have your platform adopted - you'd have to promote it and work at it. More work!)

BTW: your art assets are copyrighted automatically. You don't have to do anything (in the past this was necessary in some jurisdictions).

The vast majority of companies choose not to open source their products. It's therefore hard to draw conclusions from what happens to those that do. Please forgive me for this parody, it's for illustration purposes only: can you find just one example of a company that failed or was harmed because they exploded a hydrogen bomb in their head office?

balsamiq on piracy: http://news.ycombinator.com/item?id=337821

esr discusses id's decision (near the end) in the section "when to be open, when to be closed": http://www.catb.org/~esr/writings/cathedral-bazaar/magic-cau...

Thank you for the insightful reply.

Sorry about the inflammatory "hydrogen bomb" example - I couldn't think of a better way to express the thought (though maybe that's no excuse), but now I can: when looking for examples of companies that were harmed by opening sourcing their product, it's not the number of examples (find just "one"), but the proportion harmed, of those who open sourced their product. The thing is, not many companies have done it.

Netscape open-sourced their browser... that didn't work out too well for them (but there were other factors). Several companies are based on open source (Firefox, Red Hat), but it was open source before they came along. Some companies do open source their product, once it's old (your id e.g., there's also ghostscript). Cynwin open source their stuff, but they do it by selling services.

BTW: as the poster you replied to: I began as open source, but switched to closed source. Why? I found it very frustrating and disrespectful when people said they would buy a commercial license (I dual licensed), and then took three months to buy it. I clearly wasn't a priority - they didn't need to do anything; I was dependent on their goodwill. (I wouldn't have minded if they didn't tell me they were going to buy until later - I just found the waiting very difficult. Maybe this is a character defect, or I wouldn't have been affected if I had many many sales going at once... ). Anyone, I switched to closed (at the same time as a complete rewrite that was needed anyway), and immediately started getting several times as many sales.

Open source is tantamount to giving it away for free - make no mistake. (you've said you don't want to give it away for free; and that you don't believe open source necessarily means that - this is just my opinion here) You need a business model that works with that, such as: sell a service/maintenance; sell add-ons/upgrades; advertising (e.g. TV; google; firefox).

Your point about "pay it forward" gestures of goodwill - or just plain doing something worthwhile and good - does not mean you have to give it away for free. If you have created something great, or made something people want, or improved people's quality of life, or made something better, or righted a wrong, or fixed something that was broken, of helped people to do what they need to do - you have done something that is good. It's not just a gesture of goodwill, it's actual good. I think the focus needs to be on making something worthwhile in the first place - price comes afterwards.

I think their are two very basic pricing concerns, with respect to social good: (1). price it low enough so it's worth buying for the values it gives, and it's affordable for anyone who needs it. Too high, and you prevent some good you could have done - you can have different prices for different value given (e.g. per seat, functionality); and for different people (e.g. zero for charities); (2). price it high enough so that it's worth your own investment in doing it, and that you can afford to devote the needed time to it without starving etc (same arguments as above).

Within those two limits, anything goes, including giving it away for free if that helps.

Lastly, an exception: another way of doing good is just by price - Microsoft's DOS was cheaper; PC clones were cheaper than IBMs; web apps are cheaper than desktop apps. Often, a new conception of the whole business can reveal a cheaper way of doing it. These things are good because they make them affordable to more people, for more purposes. I'm not sure on the goodness of making something even cheaper that is already affordable to everyone... I guess it is good, but not as good as making it cheap enough to be worth buying in the first place; or, more importantly, creating the valuable new thing in the first place. Maybe it's just because I think the latter is exciting and cool.

for one discussion of "be good": http://www.paulgraham.com/good.html

I found it very frustrating and disrespectful when people said they would buy a commercial license (I dual licensed), and then took three months to buy it. I clearly wasn't a priority - they didn't need to do anything; I was dependent on their goodwill.

That is probably just the length of time to crank the paperwork through at their end. At some companies, taking only 3 months to buy something is considered quick! I appreciate it was frustrating but I'm sure no disrespect was intended.

Thanks for the reply. I'm sure you're right that it's typical paperwork delay - it just doesn't work for me. Unfortunately, this attitude of mine will prevent me from dealing with some large companies (with large cheques). I will think about what you say.

However, switching to closed source did fix this problem, and I was still getting sales from large companies.

I don't think open-sourcing an entire product is a good idea (at least on launch). However, open-sourcing components (such as your internal tools, file specs, libraries, etc) can add as much value to the open source ecosystem while preserving your competitive advantages.

In the case of crypto/security, open sourcing your algorithms and protocols can be a big win. You get free vetting of the security of your system. If you use LGPL or another suitable license, then you can still keep other parts of your product closed. Those are the parts that actually give your product "polish" and add the most value.

(I am considering doing this for a password vault app.)

While you mention that many of the benefits of keeping the source closed are either false or unimportant to you, you don't really give any compelling reason to open the source.

Your argument for open sourcing your product is simply "why not?" rather than "why" :)

Followup: I may be wrong but it seems that you're more concerned with the ideology of open source. Not that there's anything wrong with that...

Yeah, you're right. I'm a big believer in "pay it forward" style gestures of goodwill. If you contribute to others, you are generally rewarded for it in the long run (sometimes in unexpected ways). That's happened to me enough times for me to believe that there is real value to be had in helping people.

But enough generalities. Here are a couple reasons in favor of open-sourcing your project:

- Free bug fixes. If a bug is affecting a number of users, then some users will at least feel like they can do something about it. That is a very tangible benefit because there's no faster way to lose a customer than if they discover they can't use your product for some mostly-trivial reason after they dropped $50. But for desktop apps, it's more likely that a user will just tweak the product slightly to suit them. They might even have fun with it.

- Free marketing. Even though the tech crowd is not nearly as massive as the consumer audience, techies are probably the most valuable type of customer. They will help build a community around your product by answering other users' questions, for example. And what better way to attract people like us than by offering us more control?

That said, the primary reason is to help people. When I was first getting started with programming, it would have helped me tremendously to see how an "industry-strength" game engine worked, for example. I personally like the idea that a novice might cut his time spent on learning down by half or more. All they need are real examples to tweak and to build upon. So if there aren't any reasons to keep source code secret, then I'm going to share it in the hope that it will inspire someone, somewhere. How cool would it be to have access to all of Facebook's source code? And Facebook wouldn't be endangered by that at all, because its value is in its content, not its source code.

As the creator of a reasonably complex open-sourced research simulation code, I can point out a drawback with open-sourcing: The more successful you are, the more emails you will get with questions etc and responding to them can easily take quite a lot of time away from your real work. Of course, in that this is an increased incentive to document your code extensively, it might be a good thing, too... ;-)

responding to them can easily take quite a lot of time away from your real work

After writing a book about an open-source project, I get a lot of questions by e-mail. My usual answer is "Ask the mailing list."

This is much better for everyone -- the answer becomes google-able, and they can draw from the experience of the entire community, not just me.

Yes, I think the answer to the "too much mails" problem is opening a mailing list or forum. It works for medium to big projects, but for small ones I'm not sure it would be enough.

Cool! Which simulation did you create?

I couldn't agree more that it's very beneficial to thoroughly document your code. If you're curious, here's a snippet I wrote yesterday to tokenize a .X geometry file: http://pastebin.com/f74646d17 .... that type of aggressive source-code documentation is how I write all my code. My process is basically to write each function's entire algorithm in English, then to write the actual code. The result is "free" source code documentation. I also don't lose any productivity doing that. Just the opposite... I'm certain that it's more productive to write each function in that way, because I am constantly spotting bugs while writing the pseudocode that I otherwise wouldn't have caught until runtime. It's very easy to "misspeak" in C++. It's not a very good language for communicating concepts. But English is, and so by describing the same concept in both languages, you end up simplifying your ideas more than you would by just "speaking in one language". The result is usually a much simpler algorithm.

Unfortunately, the misconception that "writing comments makes you less productive" combined with the (valid) argument that "good code documents itself" makes this style of writing source code less than popular. But oh well, at least I can benefit from it, and maybe someone else will try it.

I agree that it's very easy to "misspeak" in C++ or C. It's even easier to mislead or obfuscate what's going on (i.e. write correct code that others can't easily understand). So this sort of documentation is great and I applaud you for adopting it.

That said, the fact that such extensive documentation is necessary (or at the very least nice) suggests to me that you'd be better off using a cleaner, more powerful <http://www.paulgraham.com/power.html>; language that: (1) documents itself to some degree through more elegant syntax (2) takes care of low-level tasks that clutter and confuse otherwise clean code (resource allocation, memory management, etc.) and (3) requires less code to do the same thing, which means fewer lines to document.

There are a bunch of great languages out there that are far more powerful (in terms of abstraction) than C++ or C. Haskell, Lisp, ML, F#, to name just a few. You might consider checking some of these out if you haven't already.

Be very careful about open sourcing a game, especially if it is multi-player.

Because people cheat. A lot.

Yeah, very true. I have an idea about how to detect cheaters, if you want to hear about it. Basically, I think the way to detect whether someone is cheating in an online game is by applying spam filtering techniques.

First consider that every server-side game state (like a player's position in the game world) can be considered a statistic. So for example, a history of a player's speed at points in time would have certain maximums. (i.e. "players don't usually move at 500MPH. And if they do, they only do so for very short periods of time.")

Therefore, by definition, a cheater is a player whose statistics are significantly different from the norm. A speedhacker could be detected because his max speed would become way above the "typical" maximum player speed over a significant amount of time (like, he begins to run at 500MPH for 2 solid minutes). So we can then boil the problem of detecting cheaters down to evaluating probabilities (which is the same way spam email is identified).

This type of system would also handle the case where there's a game bug that accidentally causes players to run fast -- the system won't accidentally boot everyone, because everyone is now running fast, so therefore that fast movement speed becomes "normal". Now let me point out that the very same thing could be accomplished much more simply by just writing code to boot players when their speed exceeds a certain hard-coded maximum global speed, then being very careful to not introduce a superacceleration bug. However, that's special-case code, which should typically be avoided. It also doesn't take into consideration that a player's speed could legally be significantly accelerated for a short period of time by physics, for example, or for a number of other reasons. The idea is to analyze samples of a number of different game statistics, then boot any player whose maximum statistic value significantly exceeds the normal maximum value over a significant amount of time.

This type of system would prevent a situation where a cheater discovers how to manipulate the timing variables in packets, then tricks the server into thinking he's moving faster than he actually should be. (I believe Counter-Strike: Source is still affected by that particular speedhack problem.) We would also be able to detect and ban players using aimbots, because it is very statistically significant for a player to murder 20 opponents in 10 seconds, for example.

So if people modify the client to cheat, the server will identify and ban them. Godda love probabilities and statistics.

I think it would still be hard to catch cheaters who pushed the limit in ways that are difficult to measure, for example, 'map-hacks', or 'aim-bots'. How can you tell if someone is cheating or is merely so good that they have a preternatural sense of the game? And if people know what the statistical checks are, by virtue of them being open source, why can't they push the limits while still staying well away from what's detectable as 'bootable cheating?'

They could. And cheaters using "sensor hacks" or "map hacks" can't be detected server-side because those hacks just provide information to the cheater client-side, without actually affecting gameplay directly (they don't alter the cheater's speed, aim, etc). But the way to beat those type of hacks is to minimize the amount of information the server sends to the client. If an opponent is just around the corner, then that fact shouldn't be transmitted to the client until the client is just about to turn that corner. So the client-side "sensor hacks" aren't very useful in that regard because the cheater isn't actually receiving extra information until it's basically too late to act on it.

But here's the thing. The goal is not to detect all cheaters. That's not really possible. The goal is to eliminate all seriously disruptive cheaters. For example, the guys who run around with a speedhack plus an aimbot and annihilate every opponent in just a few seconds. That would quickly ruin the game, so those are the most important type of hacks to prevent. And I could just let the community report each offense then ban the cheater afterward, but that doesn't really scale, and at that point the cheater might have already ruined the game for a lot of people. So it would be advantageous to implement a way to detect cheaters realtime.

The server-side anti-cheat code would remain closed-source, so it wouldn't be known which probabilities were being monitored. Other parts of the server code also have to remain closed-source, actually, because otherwise people would be able to examine the game's rules to gain tactical knowledge. (i.e. a player could read the game rules source code to learn where and when creatures will spawn, then that player would go around and camp all the right spots to gain maximum benefit.) However, that's a fundamentally different motivation for keeping the source code closed. At that point, it's not closed because we're worried about the competition; it's closed to ensure that the game will stay enjoyable. So that type of thing doesn't seem like it would go against the spirit of open-source.

Consider this -- if people are cheating by pushing their statistics to the point just before they would get banned, then those cheats don't actually impact other players very much, because the cheater's statistics are still within acceptable ranges. At that point, the cheaters are using the cheats so infrequently that they don't statistically affect gameplay. So in that regard, it isn't necessarily worth the extra effort to stop them, because the goal of enforcing reasonable gameplay statistics has been achieved, so the overall game experience is mostly fine from other players' point of view.

Also, your blog posts are great. Please keep on writing them. One specific part of your writing I'm interested in is this:

"Increasing my effective working memory has an impressively super-linear effect on my problem solving ability, how fast I can learn things, and also how much fun I’m having."

I have working memory problems, so it would be really fantastic to learn how you improved yours. I didn't know it was even possible to do that.

My working memory improved with a simple idea: get other things to remember for me. :-) Paper and machines do a great job. Tabs in particular are a way to expand one's working memory; the browser can keep track of what you meant to check out or do, leaving you to think about other things. This takes discipline, though. Also, I find I can store some things in muscle memory; if I've copied some text for example, I feel like I'm holding something.

With deliberate practice and the right techniques, people can learn to expand certain areas of short term memory by associating it with other, more solid knowledge. This used to be called 'chunking', but the process seems entirely different. There was a paper (referenced by Ericsson et. al. on expertise) on a group that increased their digit span memory from 7 to 80 in just a few sessions.

Some people (e.g. Ericsson) think that with deliberate practice and effort, experts route around the standard short term memory entirely: in a blink they can remember information just presented to them, in a field that they know about, in the long term. For fields outside their expertise this ability doesn't kick in, however.

For other readers of this thread, there's also this, apparently.

"Improving fluid intelligence with training on working memory"

http://www.pnas.org/content/early/2008/04/25/0801268105.full...

" The source code is just the manifestation of those decisions. In other words, the product is valuable because of its design, not because of the source code that describes that design. "

In many cases, this is true. However, there are many types of software (and, games being on of them) where its not true.

You may be able to see how the graphics are rendered on screen. But without the source code, you have no idea what kind of clever hacks and tricks the author put into the engine to make it work. These kind of things are invaluable and in many cases (like Id software) are what separates their game from others.

Id Software, incidentally, does release the source code to their game engines, but only after enough time has passed that it doesn't make economic sense to license the engine out.

Great food for thought. But I would not so lightly consider suing as the answer to anything. It's really really expensive, it distracts you from what you really want to be doing and it grinds your nerves to a pulp.

Yeah, that's a great point. I'm just saying that if someone clones your product and it actually harms you, then you do have a way of putting a stop to that, even if it's an expensive fight.

I do play computer games, but I really wont care about the Source. How many of your users are actually going to see it? Look at Apple, they have proved that they can be propriety, yet successful.

i don't see how distributing the code of your engine could aid your game be more successful.

definitely source specific. innovative algorithms stay private for good reasons

Really? Which reasons?