Hacker News Reader

Comment by memset | Hacker News Reader

The corresponding reddit thread is quite insightful:

"It is far cheaper to make one very good chip for the highest market, and modify it slightly for lower end markets...

Now, this is the part you hate: this is the only viable alternative. There's no point demanding different designs for different market segments as that would significantly increase the cost you, the consumer, pays. You can demand chips be sold at lowest prices without being fused down, but then the company eats its own market, becomes unprofitable, and goes out of business at worst - or, at best, doesn't make enough money to fund further r&d as much as they want, which again hurts you, the consumer."

http://www.reddit.com/r/linux_gaming/comments/1aj3n3/not_coo...

View on HN

Here's a key insight on this issue that the general public seems to be unaware about: the most expensive thing in workstation cards like Quadro and Tesla is warranty and support. This is also the reason that they are clocked down, they have to be more reliable than a gaming gpu.

Gaming GPUs have some parts fused off. These parts are not essential for gaming performance, and powering them off with fuses gives better thermal properties, which allows more gaming performance. These things run HOT.

Finally, this hack does not really make your card into a Tesla or a Quadro, it just fools the driver with a bogus PCI ID and enables some software features that are disabled. It does not enable hardware that has been fused out.

This is key- and I think that as long as you are just using it to get triple monitor support like this guy did you will likely be ok. However, if the software you work with is going to rely on some fused out feature of the Tesla or Quadro you are just risking frying your card unnecessarily.

I dont understand why users think that this is an evil practice. Its HARD to build hardware, and just like software engineers want to have reusable code, hardware manufacturers want to have reusable silicone. You are paying for the features you want. If you want a feature set that isn't mainstream and "workstation" capable, you are going to have to pay more. If you just want to push pixels quickly, then of course they will disable some high end functionality and sell it to you cheaper.

That said, I applaud the fact that they are digging in and learning to mod these things.

The analogy to software is a good one. Often the software is shipped full, and you unlock features as you pay for.

One thing to keep in mind, is that author mentions that the same card does not perform equivalently in Windows and Linux, specifically multi-screen support under Linux. From the forum post:

"I own a NVidia GTX 690 which I bought for two reasons, gaming, and multi monitor setup for work, NVidia made it very clear that this card would drive up to 3 screens in 2d, which it does quite nicely :-+... under windows :--! The tight asses have decided that if you want this feature under Linux you have to get a Quadro which has Mosaic support :palm:. So naturally I decided to look at how mod the card, as the price difference is over $1000 between the GTX 690 and the Quadro K5000 (same GPU) and, get this... the K5000 is only single GPU and clocked some 25-30% slower then the gaming card, what a joke"

It might be that he had expected 3 screens under Linux but didn't get it, so he set out to "fix" it. It seems a bit excessive to expect someone to spend $1000 for a card performing 25-30% less just to get Linux support :)

Apparently triple monitors on linux with nvidia cards can be achieved without this hardware hack.

Source: http://www.reddit.com/r/techsupport/comments/13vkup/3_monito...

I have a GTX680, and it supports three monitors out of the box using the proprietary Nvidia drivers on Linux. No hacks needed.

I seriously do not understand how this issue is repeatedly brought up. I've been using multi-monitors on linux with nvidia-settings for more than 10 years without any issues.

It's also worth noting that the electrical performance of your cheaper card's chip may not be as good as the higher grade chip, so may overheat more or have a failure earlier in it's life.

On chip lines like these, there's usually a process called "Speed Binning" where basically when you test the performance of each chip, and you put the higher performing ones in a different "Bin" than the slightly less efficient ones. You then sell the super high performance ones for a higher price, or put them in the more expensive product lines, as they will be less likely to fail.

i.e: Of all the chips that don't fail their tests and are rejected, 85% are of C grade performance, 12% are B grade, and 3% are A grade. Intel does this to get the "Extreme Edition" chips, and i'm assuming Nvidia does this to select the chips for their higher grade product lines.

I had no idea that the manufacturing variation among chips is large enough to create performance tiers. Pass/fail I get, but this sounds like more extreme. Like, the difference between C and fail may be such that some non-negligible percentage of chips that test into the C bin should really be in the fail bin.

It's many, many different things that can make the quality vary so much. From the purity and quality of the raw silicon itself, to the quality of the design of the chip itself, where some critical part of the chip architecture is incredibly hard to fabricate to the highest standard.

Remember that the widths of the oxide tracks within the silicon is on average 40nm these days (that's only ~400 atoms across!) or even smaller. with hundreds of process steps. One big molecule from some tiny error in the production process on the wrong part of the chip may not cripple it, but may impede performance, it's just probability at the end of the day.

With regards to potential fails going into production, it does happen, there are several test phases during production to catch as many as you can, but at the end of the day you won't get them all.

Semiconductor fabrication is fantastically expensive, new Fab plants cost several billion to build, so if you want to guarantee quality you have to pay for it.

It does make sense in that they are always pushing the edge as hard as they can, and it sounds like this process allows them to do just that. They get to sell the 'lucky' results for a premium, their 'average' for their bread and butter, and even the sub-par chips provide income.

It seems like a really good idea. It also seems like it must be pretty hard to do in a way that gives you a reliable (say) 10% emerging as 'lucky.'

When the PS3 first came out, they were having really bad yield issues. The design of the chip is one slowish "normal" processor, plus 6 "synergistic processing elements" which were really fast little vector processors. Well, not exactly 6... If you look at the chip http://www.trustedreviews.com/Sony-PlayStation-3_Games_revie... you'll see 8! OK so one was reserved by the OS for a hypervisor that would run in the background all the time and was not available for mere mortals to access. But that still only explains 7. Turns out they just disabled one of the SPEs. They tested each of them and if one happened to be broken, they would pick that one to be disabled. This N+1 redundancy improved effective yields even though a lot of the chips were still broken!

To be fair, those percentages I gave, I made up for that example. Though I'd imagine they are something close to that. But it depends on the chip, the market and the company selling them.

You design those percentage bands, generally to the size of the market you're aiming the chip tiers at. The silicon will be as good as they can get it generally, nobody wants to push bad products out. Recalls and returns probably cost more in the long term than failing more chips and suffering a worse yield.

However if only 0.5% of your customer base is interested in paying more money for a faster chip, you only cream off the top 0.5% of chips.

Keep in mind that chips can be tested at different frequencies with different amounts of input and expected output. So that C bin could have a perfect test score under certain reasonable conditions.

Clever little hardware hacks gives you oodles of geek street cred, but it may mean putting up with occasional bizarre behavior so that cred is well-earned. When silicon fails, the apparent effects can defy all logic. We can be talking about a logical AND that does something else entirely <0.001% of the time.

They do the same thing with regular electrical components. Resistors for example have tolerance bands so they make a ton of them and then if it fits in the 1% range it gets marked with the 1% color band and sold for more than the 5% tolerance resistors. It's easier to change the marketing strategy than the manufacturing process.

Reminds me of all those guys that were arrested for selling modified modems to give speed boosts to internet subscribers. The ISPs run the same pipes to everyone's house, they then check the device you're renting and tighten the spigot.

I'm fine with this as a business necessity, but I really wish someone would pitch a service that only throttles me during peak times. I feel like we're tossing a bunch of idle capacity down the drain.

The big difference between the graphics card modification and the modification of those modems is that unlike the Internet connection, you're stealing from nobody when you modify the card.

If you're going to spend the effort to modify the card to behave as what is described in the hack, then it is really no loss to the manufacturer as they had already made a sale to you and you're forgoing your warranty in the process, so they will unlikely need to spend any further money on you.

Contrast this to a cable modem modification where you modify the uploaded file to allow for faster speeds, to the detriment of others. People who legitimately pay for those speeds you've effectively stolen could be hindered as a result. You're stealing from a service in this case.

So I don't think your comparison is fair here.

A better example would be those who have older video game consoles and are tapping into the RGB lines to produce a video signal superior to of what is offered out of the box. The manufacturer never intended for this but physically it allowed for it due to different markets or how the video was processed to begin with.

Tangent: To anyone wondering about the "RGB lines" comment, apparently it has something to do with SCART cables ( http://en.wikipedia.org/wiki/SCART ).

Here are some screenshots: http://www.reddit.com/r/gamecollecting/comments/mcmsz/my_sne...

Found this out when I was wondering what he was talking about...

There's that and then there's this: http://www.made-by-bacteria.com/viewtopic.php?f=20&t=148

The SCART method is fairly straight forward, but getting at it on other non-SCART capable consoles is something else so hence "RGB lines". :)

The problem with such things cross-financing.

They give free/cheap stuff to poor/normal people and expensive stuff to rich people.

Then the expensive stuff pays for the cheap stuff.

It's like socialism for capitalists.

It's just a situation where capitalism doesn't lead to the most economically efficient solution.

The company and the consumers are rational, yet for no additional cost, everyone could be benefiting from better hardware.

Unfortunately I don't have a better system to propose :)

It is a form of price discrimination. In a nutshell, price discrimination is always better for the producer and results are mixed for consumers -- sometimes it leaves them better off, and sometimes worse off.

The claim that this is not economically efficient isn't so straightforward. In particular, your counterfactual isn't "at no additional cost." It would reduce revenue to the card-maker as no one would pay the higher price for the more expensive cards. If the producer can only charge one price, then it would be somewhere in the middle, which would hurt the consumers buying the low end card. Worst case it would make the entire card unprofitable and it would never get produced in the first place.

I think nobody would call nerfing high-end chips for people with higher price sensitivity an optimal solution, but something gotta give. Either there's a lot of consumer surplus (cheaper prices for better products), or there's a lot of producer surplus through market segmentation (bigger R&D budgets, better products in the long term). You can't have both.

This problem is more obvious with software, by the way: higher-end SaaS plans may cost marginally more (storage, support) but they're really mostly about price discrimination: getting people who can pay more to pay more even if you could provide all your customers with the top-tier experience.

The biggest problem with deciding whether a given system leads to "the most economically efficient solution" is in deciding what the most efficient solution is. Another way of viewing this is wealthy business users subsidising less wealthy gamers - which is a pretty good thing IMO.

Why is that? The major cost is the one-time cost of designing the chip, not producing it. Software is even more extreme: the distribution costs are essentially zero. That doesn't mean the most economically efficient solution is for the software to sell for nothing.

Say you have 2 audiences: Gamers and professionals.

For gamers, a low performance value yields a value of $10, and a high performance value yields a value of $12.

For professionals, a low performance value yields a value of $5, and a high performance value yields a value of $20.

If nVidia sold a single design to all of them, with a high performance value, then gamers would be willing to pay less than $12, and professionals would pay less than $20.

If they want to sell to the entire market, they have to price it less than $12, and lose around $8 from each professional buyer.

However, if they sell 2 designs (e.g: one a cripple of the other), they can derive around $10 of value from each gamer, and $20 from each professional.

The reason this is inefficient, is because for each gamer buying the crippled design, there is a loss of $2 of real value. This loss is a real value loss that could be avoided if there was some other way to extract the actual value for gamers and professionals without crippling the product.

In short, the inefficiency is not that the price is not near the production cost, but that the intentional crippling of the product (necessary to extract maximal value via market segmentation) is causing an actual value loss in the economy.

However in your latter scenario, the net loss is only 2$ versus your original scenario where the net loss is 8$. (lose 8$ per card to the perfessional, versus gamer losing 2$ of performance)

So its socially optimum to continue to do what Nvidia is doing ;).

If they sell at a >4:1 ratio in favor of the gamer version, it is still suboptimal.

If I had to guess, I would say their gamer card prices are by and large good approximations of the value they provide. No idea about professional card prices.

In the original scenario, there is no economic value loss, there is only monetary loss for nVidia. i.e: the surplass 8$ value are captured by the professionals rather than nVidia.

That's not really true, for small runs design time is a significant cost, but at scale it's mostly a question of manufacturing and capital costs.

Consider, Intel produces a wide range of CPU designs specifically because production capacity is what is most expensive. Video cards are something of a special case because they are highly redundant so companies can sell highly damaged chips as slightly different models. However, demand rarely matches the rate of defects so often chips are sold below capacity.

What confuses the issue is 'pro' cards that may preform worse than 'gamer' cards, but cost a lot more. However, that's more a case of different needs. Consider, stability is a lot more important to the 'pro' market which down clocking provides. Toss in some driver tweaks and a small market and you end up with a high priced product that does not cost more to manufacture.

Not necessarily. The people who were buying the high-end version with all the features enabled would then be paying a lot less so the total revenue is significantly reduced. Of course they could raise the prices but then the people who were buying it at lower prices for just the features they wanted will suffer. The price increase might be so much that the lower-end consumers just stop buying it altogether, and then they have to raise prices even further to make up for all the decreased revenue. And then the cost per person will actually be even higher since there are fewer consumers but presumably the total cost of production is the same.

Look at it this way. By decreasing prices in exchange for disabling some features, they increase the total number of consumers. With more consumers the total production costs can be distributed amongst more people. Which lowers prices for everyone, including the people paying for all of the features.

There's a "law" somewhere that says if a vendor can segment a msrket easily it will. Hence academic software pricing, lunch menus, military discounts for movie tickets, and so on. If there's a simple way to segment a market it will always be profitable to do so (unless you somehow piss off one of your segments, I suppose).

Academic Software pricing and military/age discounts for tickets are direct price discrimination, where you can identify groups and prevent any arbitrage.

nVidia chips are a classic example of indirect price discrimination, where you can't prevent arbitrage. In tech this is common because both hardware and software have huge upfront costs (new fab design, coding and testing software), and it's cheaper to produce one and cripple than it is two different designs.

Without the "high- margin" fully unlocked segments, businesses might not pursue projects at all, or greatly reduce the scope and budget for R&D. It feels wrong because humans are much more wired to feel loss than unrealized gains, but it is the best solution for everyone.

Don't you feel a little stupid for bringing up the term "economically efficient" when you don't know what you're talking about?

"Economically efficient" is not equivalent to "the cheapest possible goods"

Could you elaborate in a less rude and more informative manner?

No.

Lot of industries are able to survive without extensive market segmentation. Why should it not be possible for HW manufacturers?

And lots of industries aren't. Why should it be possible for HW manufacturers?

It's not relevant that some industries have different practices. It would be like asking why can't manufacturing let people work from home, since it works in programming. It's because they are different industries. Their attributes, particularly in terms of R&D and capital costs, aren't on the same scale.

Should we force our industries to survive, or should we allow them to charge what people willingly pay (for non-essential goods or services)?

Because it's about growth, not profitability.

Could you (or someone) expand on this?

Exactly, binning is good for the business AND for the consumer.

When "binning" refers to the practice of putting premium prices on parts that test as being capable of operating at higher clock speeds or lower voltages, that's good for the consumer. When it refers to die harvesting - selling the chip with a defective section disabled - that's also good for the consumer. When it refers to crippling a chip that has already passed QA, it's a symptom of insufficient competitive pressure. The best example of this is when Intel offered a $50 software upgrade that would enable HyperThreading and more CPU cache. What NVidia is doing here probably isn't as bad - the premium associated with Tesla and Quadro parts goes toward driver features and QA that is irrelevant to the consumer market.

> it's a symptom of insufficient competitive pressure

Exactly. That's as if fast food chain just before serving food to customer added ingredient that would made food taste worse (and or be less healthy) in order to be able to sell you same food without this ingredient at premium price.

We don't see such behavior because in fast food industry we have heavy competition. Not so much in GPU market where two companies have most of the market and entry barriers are high.

When it refers to crippling a chip that has already passed QA, it's a symptom of insufficient competitive pressure.

I don't know, as I'm familiar with it early in the development of a process node they sell every top-bin-SKU they can make, and as the process matures they just have too many top-bin parts. I can only guess your argument is that in a competitive market they would start dropping the price of the top-bin SKU to move them, but that reduces the overall profit of the venture which may drive them to increase initial pricing, if profit margins are thin enough.

Basically what I'd argue is that the pricing is the symptom, not the down-binning. Down-binning exists to maintain & stabilize SKU distinction and pricing, as yields change over the life of a product.

When a working chip is crippled, that's a destruction of economic value. It may be offset in the long run by helping sustain Moore's Law, but that claim is very much in need of evidence. You have to actually consider what the computer market would look like if we could now buy the flagship CPU from 3-4 years ago for little more than the marginal cost on a mature process. That level of performance is good enough for most users, and could be available for ARM kind of prices. Imagine if the Raspberry Pi were twice the price but powerful enough to be your primary machine.

consider what the computer market would look like if we could now buy the flagship CPU from 3-4 years ago for little more than the marginal cost on a mature process.

If that was what the market wanted, don't you think that is what the market would get? I think the trouble is you are thinking "the marginal cost on a mature process" would be a few dollars.

You're right, the silicon would be a few dollars- but after testing and packaging and all that jazz, I believe most mainstream high performance desktop CPUs have a marginal cost of around $30. (Why not cheaper, like a 10MHz ARM chip? Package is expensive, due to cooling needs and pins for power & DRAM interface) So, perhaps you bring it to market for $40.

This chip you're selling probably performs like a mid-range part in today terms, in the $100-150 range. But when you consider TCO due to power draw and cooling, the numbers start to get closer.

I haven't carefully laid this out on paper or anything, but point being, if such a chip would sell so well, why is nobody doing it?

It becomes much clearer if you increase the contrast by comparing today's parts to those from a decade ago. The clocks are similar, but the performance has come a long ways due to improvements in IPC, multicore, updates to the memory interface, etc.

On the other hand, high-end CPUs on a warehouse shelf are of little economic value to anyone.

> ...it's a symptom of insufficient competitive pressure.

The extra value a producer is able to capture from price discrimination due to a relative lack of competition can be thought of as one of the ways in which the deal between producers and society called "intellectual property" functions to shift rewards for innovation toward to the producer.

Something thought "insufficient" for one end, might be in proper indeed for some other end.

It's good in the sense that it's not clear there's a better alternative, but there is a painful point with a significant part of the market using a crippled design and losing value -- just to make the market segmentation possible.

If there was some other way to do the market segmentation: those who derive $X of value from the chip pay $X, and those who derive $Y pay $Y, without the crippling, then more economic value would be derived.

An example alternative for this approach could possibly be funding such developments with income tax. Then everyone can gain from the benefits of an uncrippled product, and the income tax already approximates how much value you gain from the R&D. This introduces a whole host of other problems, of course, but it does solve the crippling problem.

That's actually how we primarily fund the education system these companies rely on for their R&D hiring. Which itself utilizes a whole other layer of price discrimination for need based tuition cost adjustments.

It seems every company screws up on binning, sometimes repeatedly. AMD and Intel and nVidia have had 're-binning' issues on a lot of hardware.

Outside of smaller community of tinkerers it's not much of a trend. On the scales the companies operate it makes no difference to bottom line. Few can be bothered with learning SMD soldering.

If you're linking to a specific Reddit comment, please link to the actual comment, not just the post it's a comment on.

It's possible for nvidia to replace the resistors with one-time-programable memory ,and program it at factory. And it's cheap, it's being used in $1 FPGA from silicon blue.

And they'll probably do this in their next chip.

I don't even know if they'd bother to do this due to the handful of people who are actually going to buy the gaming card to make it into a pro card.

Most people who are probably doing this are hobbyists at best. Most professionals would probably just have their company buying the cards anyway, and this hack won't at all change that.

Actually, it being a hardware hack rather than a software hack makes it even less appealing even for small shops to try. "You want to take that new $400 video card that we just bought you and do /what/ to it?" vs. "ooh, it's just a software upgrade" (even though both have the very real chance of bricking the card in inexperienced hands).

have their company buying the cards anyway

as if the company had infinite resources and didn't want to spend less.

This isn't true for startups or any kind of small business.

Some people could turn this into a small business I guess.

Doesn't the fact that they're selling the same chip for a lower price mean they are profitable with that low-end price, too? I doubt they are just dumping the leftovers for the low-end, and taking a loss.

Also how is this any different from the Adobe scandal in Australia from a couple of months ago? I believe the Australian government even took them, Microsoft and Apple to court over charging Australians a lot more than Americans or others for the same products.

You are thinking about chip costs as if you were selling bread or cars, where the majority cost in making the product are the marginal costs of manufacture. This is not true for microchips. The marginal cost to produce any modern Intel cpu is <$40. Most of the cost of making chips is in developing the process they run on, and designing, testing and validating the chips.

When the economics of making chips are $2B down to make the first one, then $25 for each that goes off the line, you cannot measure profitability like that. They need to recover R&D, and providing more expensive chips to the professional market through price discrimination is a good way to do this.

Making the mask is really expensive. I had an external lecturer tell me it cost about 1 million usd to make a set of masks for their silicon, I assume Intel pays way more.

Once you have the mask though, pushing out chips is cheap and easy.

And the masks are just a really, really small part of the total cost. Intel has to pay for their fabs, they have to pay for the research into ever smaller line widths, they have to pay for materials research for new gate materials, and they have to pay for the massive research and development to design new, better chips.

It isn't about just being profitable, it's about extracting maximum value from every customer.

Ideally, the company would haggle with each and every customer, and get each customer to pay the highest price he in particular accepts. Since haggling is not feasible in mass markets, customers get grouped in segments, where the price is set to the maximum the segment will accept.

Clearly, if you misclassify a hacker into an expensive segment you get your hardware hacked :-)

Yep. in my view, if the product is already profitable at a lower segment, and they manage to sell it to the pros for a higher price, then the company is making undeserved money, rightly so, bears the anger of their customers when found out.

I completely disagree. Even if it was as simple as just charging "pros" more money (it's not), the idea that this is somehow "undeserved money" is pretty ridiculous in it's own right.

I can't break it to you any more easily than: You are flat wrong. Segmentation is micro-economics 101 and is an expected company behaviour.

Perhaps I can explain briefly. The "deserved"(sic) price is a value where both buyer and seller are comfortable doing business. In most cases, this is not a single value, but a range of values. At the minimum threshold the seller turns comfortable, and then there's a range where the transaction is possible and at the maximum the buyer turns uncomfortable.

Now, while for a given product and fixed seller, the minimum value of the agreement range is fixed, the maximum is dependent on the buyer. Yes, it is possible to sell all products at the minimum, but then you are giving all shared value to the buyer. Even if you are aiming for fairness, the transaction should occur at the middle of this agreement zone (the buyer buys at less than his maximum price, and the seller sells above his minimum price).

You could tackle this difference between consumers on a case by case through individual negotiation. Obviously this is not practical, so the next best thing is segmentation.

You may think this as unfair, but this is a result of seeing a half-empty glass. Segmentation allows for a company to subsidize "cheaper" products using "premium" products. If a cheaper product covers variable costs (but not fixed costs), and does not cannibalize the premium products, segmentation allows for prices below what would be possible if the burden of fixed costs had to be assigned to the cheaper segment. In industries where most of the cost is fixed (as in semiconductors), segmentation is key to achieving large volumes without compromising the ability to profit.

The product is not already profitable at the lower segment, unless the higher segment is there to help defray the capital costs of building the fab.

Chips with more defects can be sold on the low end rather than scrapped, making production easier and more profitable.

Yeap. AMD did this with its Phenom II chips. Maintain the same fab process for a 4 core processor but sell it as dual or triple core if it was poor quality. AMD motherboards famously had the feature to enable the locked down processors. Obviously your mileage would vary but going from 3 cores to 4 was a pretty significant performance boost especially for the initial cost.

A while back there was a way to unlock an AMD 6950 GPU to a 6970 as well. Had something to do with unlocking some memory modules since the cards were identical.

I think it's incredible how some people enjoy getting screwed over. I have a friend that actually thinks it's a good thing that iPhones break easily.

That seems out of context; it is fairly common for people to abuse return policies like Apple's to get free upgrades after (un)intentionally breaking their hardware.

Interesting, I just finished reading Joel's article on pricing talking about this: http://www.joelonsoftware.com/articles/CamelsandRubberDuckie...

Reply on news.ycombinator.com