luu

I work for a hardware company. Bugs are really, really, bad. If we find a hardware bug in real silicon, at best, we catch it the moment we get the first chip back, and it means that we have a multiple month delay as we fix it and tape out a new chip, not to mention the cost of throwing away all of the partially fabbed chips we've got, plus the multiple million dollar cost of a new mask set. At worst, we have a recall [1]. We take testing very seriously, and we do a lot more formal verification than most software companies.

The only things you can be sure about are things that have been formally verified [2], and the list of things that you can formally verify is tiny. We formally verified our adder. It took months. Then we did the multiplier, which was much harder. It took about the same amount of time because of the experience we gained doing the adder, but it wasn't easy. Division took a lot longer, even with the experience of doing addition and multiplication. To think that we can advance the state of the art of "things that work" from something like a multiplier to a complex piece of software with "care" and our "collective will" seems overly optimistic.

Everything's going to be broken for the foreseeable future. Putting more effort into testing and less into features is a difference in degree, not in kind. It won't even prevent articles like this from being written because, if all you want to do is find ten bugs in all the software you use, that's still going to be trivial. Considering how much progress has been made in formal methods since 1970, I expect that finding 10 annoying bugs in all of the software I use will be trivial for my entire lifetime.

[1] Well, you don't have to do a recall. AMD had a hardware bug that could be fixed by a patch that degraded performance by 10%. Sun famously didn't include ECC in their L2 cache, which resulted in transient failures for a number of customers, and they made customers sign an NDA before replacing their parts. Guess how much people trusted AMD and Sun afterwards?

[2] Even then, you're never really sure. How do you know the formal verification process itself isn't buggy? It's turtles all the way down. I know some folks who were trying to build a formally verified OS, and they stopped using ACL2 after discovering a few bugs in it. After all, how can you trust your proof if the proof system itself has bugs? ACL2 is old and crusty, but that's precisely why it's used for more hardware FV than everything else combined, outside of Intel and IBM (both of whom have their own, excellent, internal tools). It's old enough to have great libraries. There are newer systems that have better architectures, but they don't have anything approaching the same level of library support for hardware. Yet another tradeoff of time to market vs. correctness. It can't be avoided.

Say you're an engineer who's worried that ACL2 is too buggy for your company to use. You tell your manager. She points out that maybe five ACL2 bugs are discovered every year, and they get more minor each year, as the system gets cleaned up. Moreover, none of the bugs discovered in the past three years have affected any of your proofs, and you wouldn't expect them to have an effect on any proof techniques you're going to use. So you stick with ACL2. And, because you do, there's a tiny risk of a bug. What does this example have to do with the original post? Bugs come from making little decisions like this. No single decision is sure to cause a problem, or (in a company that's serious about testing) even likely to cause a problem, but multiply that tiny probability by the number of times you have to make a tradeoff and the number of lines of code, and it's a statistical certainty that you'll have bugs.

It took fifty years of a well executed and persistent legal strategy to go from Plessy v. Ferguson to Brown v. Board of Education. Plessy lost the test case and had to pay the fine. The equivalent here is to pay $1B in fines.

I'm not saying that fear of terrorism or other boogeymen is as entrenched as racism was in 1896, but if it was that easy to change, we wouldn't have the TSA and all this other nonsense. We wouldn't be instituting policies that erode privacy, the same way the UK has been over the past couple of decades. I remember when I was a kid, and people talked about the stuff happening in the UK as if it was on the moon. Now that's normal. I would love it if that changed, if someone would put in the sort of effort the NAACP did from 1909 to 1954. But, I don't expect twitter to risk bankruptcy doing it.

Even if the founders are completely selfless and willing to throw away their fortunes, they'd probably feel that they have a responsibility to their employees to not run the company into the ground to prove a political point. I wouldn't be surprised if it's not even possible to do it, given twitter's legal obligations. The VCs might be able to step in and get management removed if they start burning money on political causes. And, if they're going to do that, why shouldn't they just donate it to starving children in Africa? Because it's not their money, not in that sense. It's one thing for Bill Gates to take the money he's made off of MS stock and spend it on causes he supports. It's an entirely different thing for the CEO of a company to use company funds to support the same causes.

I should point out that, at least the NAACP managed to accomplish something, which doesn't always happen. I have radical leftist friends. They go to protests, get beaten up by the police despite protesting peacefully, and are often charged with crimes, so that it will look like they're just trying to get out of something if they complain about the police. It's been happening for decades, and nothing has changed. It's possible to make a stand, and for no one to care.

The alternate headline, call it the fox news headline, if you like, goes something like this

Twitter CEO refuses to pay fine yet again, company is held in contempt again. Twitter's complete disregard for the legal system comes on the heels of . . .

I'll quote a few relevant snippets from the article, for anyone who doesn't want to read the whole thing.

Challenging the conventional view, there is actually a striking global correspondence between the world economic slowdown since 1973 and ever-increasing levels of educational spending. Comparisons between countries also confound the idea that more education translates into more growth. For example, South Korea is often given as an example of a country that made education a priority since the 1960s and saw significant economic growth. But as Professor Alison Wolf from King's College London points out, Egypt has also prioritised investing in education, but its growth record has been poor (4). Between 1970 and 1998 Egypt's primary enrolment rates grew to more than 90 per cent, secondary schooling levels went from 32 per cent to 75 per cent, and university education doubled - yet over the same period Egypt moved from being the world's forty-seventh poorest country to being the forty-eighth.

A retort might be that education isn't the sole determinant of growth - other factors may offset its positive economic role - but it remains a necessary one. But this argument doesn't stand up either. The rapid growth of Hong Kong, another of the East Asian tigers, wasn't accompanied by substantial investment in education. Its expansion of secondary and university education came later, as more prosperous Hong Kong parents used some of their newfound wealth to give their children a better education than they had had.

A study for the World Bank came to similar conclusions. In a paper called 'Where has all the education gone?', public policy academic Lant Pritchett couldn't find the positive correlation between education and productivity that underpins so much current international development thinking (5). Economist William Easterly summarised the study:

'African countries with rapid growth in human capital [the fashionable term for people's work abilities, especially levels of education] over the 1960 to 1987 period - countries like Angola, Mozambique, Ghana, Zambia, Madagascar, Sudan, and Senegal - were nevertheless growth disasters. Countries like Japan, with modest growth in human capital, were growth miracles. Other East Asian miracles like Singapore, Korea, China, and Indonesia did have rapid growth in human capital, but equal to or less than that of the African growth disasters. To take one comparison, Zambia had slightly faster expansion in human capital than Korea, but Zambia's growth rate was seven percentage points lower.

I believe that spending on education is positive sum. That surely isn't always and everywhere true, but I believe that it's true at the margin, considering the conditions in the U.S. The book that originally convinced me of this was _Children as Pawns: The Politics of Educational Reform_, by Hacsi. It details, along with a lot of other evidence, a longitudinal randomized controlled trial (which is very rare in educational policy) which shows a positive benefit from additional education spending (admittedly, of a very specific type, and in one particular context). There's no way to provide a good summary of the evidence in the book, so let me just say that my opinion is that while the evidence is tenuous, the overall weight of the evidence indicates that the gains from education in tax revenue are greater than the costs. If you like, you can check out the book and decide for yourself. If you agree, then government funding of education seems like an obvious solution. And, of course, the overall gains to society are much greater than the increased tax revenue.

I don't mean to say that our educational system is particularly good. In fact, I think it's very inefficient in many ways. That doesn't preclude the previous conclusion, though.

The second reason is that, in the model above, college administrators capture much of the value added by subsidies. Some gain goes to "good" students, and "bad" students are severely penalized. I believe that's apparent from looking at the practical results, and you can see that in the model by adding student ability and graduation rate to the simple model above. You get an even more extreme result if you add types of degrees (hard vs. easy, with hard degrees resulting in more compensation. Typing out the entire "proof" would be tedious, but I can do it, if you disagree with the result of the model, given the premises. But, in the end, this is a philosophical difference. I don't think that we should be subsidizing college administrators (who are already much better off than average) and really good students (who, on average, will do fine in life anyway) at the expense of bad students (who, IMO, could use some help). I understand that a lot of folks would prefer that people are rewarded purely on talent. Having subsidized loans with a random probabilistic outcome magnifies that, so perhaps you think that solution is fair and just. If that's the case, and you're not convinced that spending on education is a net positive, then there's no hope my line of reasoning will convince you, and we'll have to agree to disagree.

I agree that incentive compatible solutions are important, but I don't think that creating a system that's efficient (in the sense that an econ 101 student would use the word) is the only thing we should consider.

P.S. Your original comment has been downvoted, but it wasn't by me. I disagree with your position, but I don't think you're arguing in bad faith or presenting even presenting a fallacious argument, which are the things I would downvote someone for.

The NPV of college is $100k. The marginal student can scrape together $10k to go to college. The marginal cost of a student is less than $10k for the college. How much will the college charge? $10k. What's the benefit to the student? $90k.

What if an $89.9k subsidized grant is available? Now the college can charge $99.9k. What's the benefit to the student? Still $90k.

Now lets say an $89.9k loan is made available. The college can still charge $99.9k. What's the benefit to the student? $.1k. This is actually happening (though it isn't quite so extreme). The existence of loans is allowing colleges to capture more of the value of college from student.

The real world is actually worse than this simplified example in a lot of ways. Many degrees aren't worth it (from a purely monetary perspective), and the result is probabilistic, but student loans can't be discharged in bankruptcy, and payment is up front (w.r.t. a person's career), so the student takes on the risk, but the college gets the reward regardless.

What's so great about LabVIEW for spetroscopy? I ask because, as part of my MS thesis, I built a fourier transform spectrometer. After messing around for a few days with some LabVIEW code someone else had written for a similar task, I wrote the control code in C in a couple of hours, including the time it took to learn the (deprecated) LabWindows C API. I have no doubt that LabVIEW has improved since I did my thesis (almost ten years ago), but it's not obvious, a priori, that graphical programming should be better for controls.

I work in an area where bugs are very scary, so we use formal verification, and that's on top of having many more testers than developers.

From the perspective of this naive outsider, I'd would have expected FV to be worth it for security sensitive protocols. Is it that the protocols are too complex to be verified, or is it just not considered to be worth the effort?

What perils? Later in the article, lawsuits are mentioned, but the whole thing is pretty vague.

Putting aside both the legality and morality, doesn't this policy seem odd? If there's nothing wrong with what people are doing, then there's no need for this at all. If people are doing something wrong, shouldn't there be legal censure, or something? Not just a warning about some mysterious "perils of downloading unauthorized content"?

What premises could you possibly have that would make the correct conclusion "warn people 5 times and then take some vague and indeterminate action"?

In the long run, the court of public opinion is the driver of change. What makes the status quo so great that you hate it when people dissent? I don't agree with the person you're disagreeing with, but where would we be if we couldn't even discuss it?

I was 6'2'', 168 lbs, 1% body fat

When I was climbing competitively, I once had my bodyfat measured (by DEXA) as just under 3%. For climbing, pretty much any weight (even most muscle) is just dead weight, so climbers are as skinny and lean as any athletes out there, and I only very rarely ran into people as lean as me.

I've known a lot of bodybuilders and wrestlers, and even there, it's rare to see people below 3%, and I've never heard of anyone who's tested below 2% bodyfat. 1%? I won't say that it's impossible, but to be alive and functioning at that level would require you to be a genetic freak on the same level as Lance Armstrong or Michael Phelps.

Most places choose long lines, but Apple seems like the sort of company to choose to have people some people idle most of the time rather than have long lines some of the time.

I admit it's possible I'm fooling myself, but it's the way a lot of people are saying -- that I just skated by on easy courses and missed out on the material in the hard courses. I actually got my As mostly in the advanced undergrad and grad classes. Most of my Bs came in my last year, when I had to go back and take all the intro classes I skipped in order to meet graduation requirements. The math department was fine with, e.g., letting me skipping undergrad abstract algebra because I'd taken graduate matrix analysis (which requires grad algebra), but the engineering department wouldn't let me skip intro e-mag even though I'd taken high powered diode lasers (which, IIRC, required grad e-mag II).

A lot of the Bs came in "easy" classes where the average grade on exams was 85-90. I never did enough problems to stop making mistakes, so I got an 85-90 on pretty much all my exams, which is only good for a B if that's average. The classes that I did really well in were the ones where professors asked questions on exams that required real original thought; the average grade on one of those exams might be something like 30 or 50, but I'd still get an 85-90 most of the time. The one way I selected courses that were "easy" was that I chose professors who made homework a small part of the grade, so I wouldn't fail just for not doing any homework.

Someone else mentioned Ramanujan. Well, I'm not a genius, and I know that. One of my high school friends got a silver in the international math Olympiad, and he didn't even think he was good at math because his little brother was a lot better (and his little brother later got a gold). I'm not even close to that, and even those guys aren't in the same league as Ramanujan, but I read pretty quickly and I have a decent memory for abstract concepts. I think whether or not people are sure the original poster is full of crap depends on who they hung out with in high school and college. Frankly, I was the dumbest of my close friends in high school, and about average among my college friends. When I look at my high school and college friends, any of them could have done this (with the exception of the guy who never made mistakes because he was slow and methodical, even on easy stuff), so it's not obvious to me that the original poster is just spouting off, even if it seems obvious to everyone else.

The math olympiad friend I mentioned above would go through the text for his classes and do every problem in the book before the class started. That took him a few of weeks. Someone mentioned that an engineering problem set takes more than an afternoon. That's probably true for a lot of people, but problem sets in engineering and math don't take much time if you know how to do them. The limiting factor is how quickly you come up with a solution, which can be pretty much instantaneous. There are some exceptions -- I remember some VLSI questions that basically amounted to doing a SPICE simulation by hand. And most labs are, unfortunately, gated by the amount of tedious work you have to do.

On the comment that my experience doesn't count because I'm an outlier, well what's P(guy makes original blog post | not an outlier) vs. P(guy makes original blog post | is an outlier)? I don't have any idea, but neither does anyone else here, and yet every other poster seems to be completely certain that the guy who wrote the original post is full of it. I'm not even saying that it's likely that the guy is credible; I'm just saying it's possible.

I can also believe that someone who's capable of doing that would fail out, because I almost did the same thing in high school. IIRC, my GPA was about 1.6 when I applied to colleges. I only managed to get into a decent school because I was lucky enough to be from a state that had a decent state school that had a policy of letting anyone in with an ACT score above some threshold (32 or 34, I forget which). And even then, I almost didn't got my admission rescinded by failing to graduate. A lot of teachers in my HS had a policy of giving people As in their class if they got a 5 on the AP exam. I got a 4 on the AP chemistry exam, and a big fat F in the class. I'd failed enough classes that I didn't have enough credits to graduate without AP Chem. The chemistry teacher didn't know that, but he was still nice enough to change my grade to a C when he saw that I got a 4. Thanks Mr. Swanson. I don't know what would have ended up happening if I failed out of high school. I spent most of my time in college in libraries, just reading. In theory, I could have done that even without being a student, but I doubt I would have.

I wasn't quite as quick as the poster -- the total amount of effort I put into a course probably amounted to 2-3 days of work, including time spent in classes and exams. But, I knew people who were easily an order of magnitude more productive than I was; for them, it would have been an afternoon of work. If you read quickly, you can finish reading a textbook in an afternoon. The extra time I burned was the time it took to work through enough exercises to convince myself that I really got the material. For some really easy classes, I only had to do one exercise per section to convince myself that I knew it, so it ended up being just one afternoon. There are plenty of people who are smarter than I am, who could do every class that way. I don't know if the writer of the original article is one of those people, but it doesn't sound implausible to me.

Just for reference, I went to Wisconsin, which is approximately as rigorous as RIT, and I double majored in math and computer engineering, which are both comparable to CS. But, I graduated in three years and my tuition was a few thousand a year, so the total was was easily covered by an engineering internship. At a place like Wisconsin, you might as well stick around and get the degree -- after all, it's only a month or two of total effort, and you can easily make enough to cover the degree from doing internships, or working as an RA on campus or even at Starbucks. If the price tag were $44k a year, I probably would have dropped out, too.

It’s rare to see serious empirical work in SE that’s up the same standards as empirical work in modern economics (either an RCT, or a good pseudo-experiment from a clever instrumental variable), and when you do, there’s little to no discussion of how generalizable the work is. There’s often a paragraph about how the study may not generalize, which seems to be enough to satisfy peer reviewers, but isn’t nearly enough to figure out if the methodology under test has any applicability whatsoever to your setting.

[1] if you're curious, and want to look up what's in vougue, the premier SE conferences are ICSE and FSE

If you mean someone you click with, it’s easy to say “let me know next time you go Xing”, if you share a hobby; if not, you can just decide to meet to check out some interesting food cart or restaurant, and then go from there. Is that really so hard to do? I’m incredibly shy, and even I can do that much!

Historically, low-end parts have managed to kill high-end parts by selling in larger volumes and amortizing their fixed costs over a larger base. ARM sells in huge volumes, but how is selling $1 embedded chips going to help them amortize the cost of their high end stuff? Decades ago, doing that sort of thing would helped Intel fill up their idle fab time while competitors like DEC struggled to pay for their fabs, but ARM has never owned its own fabs, and selling completely different low-end parts doesn't really help amortize the cost of masks, post-silicon debug, etc. of their high end parts.

I'm also skeptical that ARM will retain its power:performance advantage if it tries to scale up performance to match low-end mainstream Intel parts (e.g., core i3). The main reason ARM has better performance:power is that there are diminishing returns to performance. Another is that Atom is a new product line, and Intel doesn't have much experience trying to optimize for low-end parts. That won't be true going forward. Yet another transient advantage ARM has is that Intel uses a high performance process, but they've said that they're going to have both a high performance and a power optimized process in the future. If anything, Intel's process wizardry should give them an advantage there, once they decide to push it. What other inherent advantages does ARM have over x86? It's a PITA to decode x86 instructions, but the ARM instruction set isn't very nice, either (e.g., look at how they ran out of opcode space, and overlayed some of their "new" NEON instructions on top of existing instructions by using unused condition codes for existing opcodes).

I wouldn't bet against ARM but, if they win, it will be because of the usual reasons: better marketing, better business practices, and better engineering, not because there's some inherent advantage that lets them be cheaper than Intel. And, by the way, the configurability advantage you mention actually hurts them in terms of cost, by increasing the fixed cost per part.

My company uses FPGAs to for verification during development (because it takes months to get silicon back after tapeout, and it costs millions of dollars even if you only want one single test chip). We make chips that are much more complex than a P3. Friends of mine at other microprocessor companies have told me that they have a similar process, though some of them at larger companies use their own custom hardware for accelerated simulation, since you can't fit anything like a POWER7 or even a core i3 on a commercially available FPGA.

The people I know at companies where everyone is explicitly ranked and rated in a way that's made public (e.g., Intel) mostly dislike the pressure it adds and the comparisons it creates. If pay and promotions are based off these leaderboards, then this has all the advantages and disadvantages of the Intel system, and there's nothing new here. If the leaderboards are just "flavor", then this is just another form of lucite plaque, and there's nothing new here.

Seriously? The advantages are legion. It would take days to write them all out. I'll just put down a couple. You don't have multiple denominations of shells, so you have to carry around a wagon full of shells, instead of a few large bills.

From a macro perspective, cash has even larger advantages. You can't conduct monetary policy as easily with a commodity currency. One reason recessions were so bad in the past (in the 1800s) is that commodity currencies are pro-cyclical, partially because of nominal stickiness, partially because of speculative attacks, and for other reasons, too.

For me, what's the advantage of replacing my credit card with a phone? My wallet becomes 1mm less thick, unless I want to carry my credit card around as a backup in case my phone battery dies. In a few years, when this becomes standard, and phones are good for multiple days, I'll be able to shave 1mm off my wallet. That's nice, but hardly analogous to the difference between paper fiat money and a hard commodity currency.

I haven't tested it myself because I got tired of working around Mono issues, and just wrote a little script that sends .fsx scripts to my windows machine, runs them there, and returns the output on my linux machine. I can see why you wouldn't want to do that in a production environment, though.

I haven't done any Itanium assembly since McKinley (Itanium 2), but their shifts had 3 extra cycles of latency. Making a fast shifter has only become a harder problem since then, because the ratio of wire:gate delay has grown; they could have a single cycle shifter in the current implementation, but it would cost them even more than it did back then.

x86 is unique among modern architectures because it has so much legacy baggage. Shifts are only fast on x86 because when Intel implanted slow shifts on the P4, it killed performance in a lot of apps that were compiled with compilers that were accustomed to having fast shifts, so that they would substitute shifts for multiplies whenever possible.

There are a series of behavioral economics studies that show, in the right circumstances, that going with your gut is better than explicitly making a list of factors, assigning weights to them, and making a decision after that (whether that decision is algorithmically determined by the list, or is a holistic decision that comes after looking at the list). The ‘gut’ method tends to be better when decisions are more complicated, and have many factors, as is the case when hiring, so I suspect that pie method is one of those things that sounds helpful but isn’t. No one knows, though (unless there’s actually been a study on this; if there has, please, correct me if I’m wrong).

An SNES would be much more annoying to do, but probably no more than an order of magnitude moreso, so I doubt it would take more than a semester, even for someone who'd never touched an FPGA before.

Isn't it likely to not even be script kiddies, but machines that are part of a botnet owned by script kiddies?

Anyway, mine's https://plus.google.com/108579573899435547873/