ShabbyDoo

I regarded my MBA classes as semester-long courses in applied critical thinking. There were few right answers although there were those which clearly were wrong. At no point did any professor say, "Make business decisions based on the values of these metrics."

A good analogy might be found in some software I'm working on now. Part of it is a specialized serialization scheme designed to produce small serialized forms of large numbers of similar tree graphs. It is easy to quantify the size of the serialized form of N million trees, and it is easy to measure the CPU/IO time required to produce this form. However, when making design decisions, I only use these metrics as one input into my thought process. If I do X, how easy will it be for others to maintain after I leave (I'm a contractor)? Does a proposed design decision limit future opportunities for this module's use? Does "brittleness" increase? Does this decision move the design further away from using Mongo/HBase/whatever in the future, or does it lessen the current impedance mismatch? The easy metrics are time and space, but the might not be the most important ones.

My MBA is circa 1997, and I shared your experience. A friend with a much "better" MBA than me (his from Harvard) once remarked, "Modeling is a great way to determine if a business idea is awful, but it's not going to tell you if you have a good one." I might argue even further that modeling can lead one to dismiss profound ideas through the erroneous presumption of a static future. It was said that the original business justification for Xerox counted the number of typists in America and multiplied by the number of mimeograph copies produced from each document. Obviously, the copy machine found many more purposes once introduced into offices -- a much more dynamic effect than most could even imagine.

Perhaps Dell's artificially static future was the presumption that shipping a laptop or desktop with a pre-installed OS would continue to be a viable business model? What if Steve Jobs had been forced at gunpoint fifteen years ago to lead Dell? What would the outcome have been? He likely would have spoken harshly about the elephant in the room -- Dell produces a commodity, and its future depends on Microsoft's continued prowess. With such an admission out in the open, what actions could have been taken which would have been nearly impossible to quantify via traditional B-School modeling techniques?

"There’s a big flaw in the bubble argument, though: things may look grim for college graduates, but they’re much grimmer for people without a college degree."

No. There's a flaw in the reasoning above. For years in the US, society has perceived that there is something wrong with a young person who did not complete high school. Want to find a good mate? Stay in school. Want a decent job? Stay in school. So, young people, fearing societal judgement and the accompanying disadvantages, stayed in school. Given these judgements, those who did not complete high school likely indeed had something wrong with them or came from families without means. So, a simple comparison of the outcomes for high school graduates vs. dropouts usually has shown significant differences. What percentage of these differences can be explained by the selection bias of the sample pools vs. actual effects of the treatment (a high school diploma)?

The same selection biases hold true for a college education. Few with ambition have considered skipping college. Without any metrics on the abilities, goals, etc. of those skipping college, we can not determine whether differences are the result of selection bias or treatment.

My Hungarian in-laws "dropped out" of high school in 1956 to come to the US. During the revolution, they escaped across the border into Austria and eventually got to the US (Green Cards effectively handed out when they got of the plane). My father-in-law attended a few night school classes over the years, but never obtained any formal degrees. He retired as the CEO of a small manufacturing organization for which he still consults -- a good outcome for anyone by most accounts. The difference between him and the average middle class American teenager was that it took a certain level of ambition and risk tolerance to decide with only a few days notice (the revolution was sudden and short-lived) to leave his home country for an uncertain future. Not entering into a degree program was an active choice -- not a passive result of low ambition.

I am annoyed with the popular press's continued assertion that education has value simply because the outcomes of those with education are better. Unless someone measures various personal characteristics and then randomizes the treatment (education vs. workforce entry), I won't buy into claims about how valuable education is.

Ironically, I have two graduate degrees.

Thank you for making your process public. In almost any endeavor where one compares his own process only to the observable outcome of another's process, he can believe erroneously that his (possibly superior) results are the fruit of inefficient labor and near incompetence. While I must edit repeatedly to produce reasonable prose, PG can just dash off a brilliant essay. Few know how may takes were required for a screen actor to nail a particularly difficult scene -- we just see the near-perfect result. Obviously, the same phenomenon occurs in software development. Look at how brilliantly Kevin Bourrillion (et. al., of course) designed the Guava API! [One of my favorite APIs from a design perspective] How many discussions over beverages with Josh Bloch were actually required, and to how many internal iterations at Google were we never privy?

It is only by adjusting our perceptions of the processes by which others produce quality work that we can feel good about our own abilities.

A random thought in response to myself:

My uncle recently retired as a plumber. He is in his mid-60's, and the physical part of the job got to be too much for him. Imagine a service like oDesk where my uncle could sit in front of his computer and be paid by the hour to consult with DIY-ers on their plumbing problems. A guy with a leaky faucet could wear a camera on his head, and my uncle could give him instructions as he took the thing apart. Let's say 1/2 hour of consulting was required @ $50/hour. My uncle would be delighted to earn even 2/3 of that in retirement, and the guy doing the work would have less applied time than if he had to wait at home to let the plumber in. Furthermore, 1/2 hour is less than the plumber's expected commute to the customer's house.

How many in-home shadow work tasks could be done more efficiently if the physical labor was separated from the knowledge aspect of the work?

Heck, sales and income taxes contribute to this. Why is Home Depot so busy on weekends with DIY-ers? Shouldn't an amateur's effective "wage" for finishing his basement be incredibly low compared to a professional with knowledge/tools/etc.? So low that he/she would gladly work a few more hours at the office rather than spending his weekend figuring out how to lay tile? The problem is that I am not taxed on my "earnings" from the weekend work I might perform around the house, but a professional is (unless paid under the table, of course). So, providing labor to myself is effectively subsidized. Furthering this is my employer's subsidy of my weekend labor in the form of health insurance, disability insurance, etc. The professional (or his boss) would have to pay for Workman's Comp insurance (at a pretty high rate for construction labor), but my weekday employer would foot the bill -- just as if I had hurt myself while out walking around the block.

Now, take into account the US's common labor arrangements -- salaried and hourly employment (as opposed to piece work, commissions, etc.). Let's say you are paid a fixed salary. The benefit of spending a Saturday in the office doing extra work to impress the boss has an uncertain payout far in the future -- one which might rationally be discounted. However, the benefit of finishing one's own basement instead of shelling out an extra few thousand in labor is tangible and much more immediate. It is no wonder mid-level salaried employees show-up at Home Depot on weekends. The lot for hourly employees is worse. For some strange reason, the US requires that they be paid 1.5x/hour for work beyond a 40 hour week. 2x on Sundays. This is a huge dis-incentive for an employer to offer his existing employees the opportunity to work weekends when business is heavy. It might be possible for someone to pick-up extra, part-time work in areas where he has expertise, but this work often is hard to come by. So, Home Depot flourishes and basements are finished with off-kilter walls.

Personally, I have had multi-month periods as a software contractor where a client said, "work as many hours as you can, please!" So, every hour not working had a real, easily-quantified opportunity cost. I would have been crazy to have mowed my own lawn. It would have been nice to have been able to purchase other "services" like a doctor's visit without a wait. I even tried to offer the doctor's office a premium to be seen in a high-priority queue, but most insurance plans don't allow such upcharges on top of pre-negotiated rates. You can probably look at my HN posting history and figure out the periods when I had effectively unlimited earning potential -- there are huge gaps where I barely had anything to say.

Darn. I have a dual-fuel system (heat pump for spring/fall with natural gas for winter), so I think I'm out-of-luck for now. My current thermostat has an exterior temperature sensor and switches between modes when the outside temp drops below a certain level. Heat pumps are energy efficient on mildly cold days but don't work very well in the dead of winter. FYI, I live in Ohio.

"why have the display to begin with?"

My guess is that many consumers have replaced thermostats before and have found the process fairly painless. Or, maybe they understand the idea of replacing one dingus on the wall for another. The idea of cutting wires in one's basement is much scarier. Your suggestion is quite rational but probably hurts sales.

Worth it if it can save you a few dollars/month and make you more comfortable in the process.

I would be willing to carry something-er-other in my wallet which would alert my home automation system (which I don't have) to my presence/location in my home. I usually leave my wallet in our bedroom when sleeping, so it would still be accurate at night. My wife might be willing to wear a small ring or something like that, but it would be a harder sell.

So, Netflix increased prices by a pretty high percentage and managed to lose only four percent of subscribers? This is astoundingly impressive. Furthermore, many of these deflections might simply be the result of forcing subscribers to explicitly choose to continue service. How many people kept paying a few dollars per month for a service which they did not use and, when forced to choose a new plan, opted out?

Let's say I was tasked with collecting and evaluating data about the selling price of homes in my community. Those interested in my findings likely are using the information to make financial decisions related to buying or selling a house. Now, let's say that one of the homes sold was not an arm's length sale -- a parent sold the house to a child for what appeared to be a very low price compared to other sales. Should this sale be included? Unlike the other sellers, maximizing the sale price was not a primary motivation. Including this data point would artificially lower average sale prices even though this individual seller's likely does not reflect his best financial option.

There are some graduates whose near-term primary motivation is not money. This is ok, but it is not correct to include those students in averages which will be used for the evaluation of the financial benefit of receiving an education from Stanford.

Yours was the comment I was going to make. Furthermore, while I appreciate Yegge's thinking, I don't see how the facts he presented are not common knowledge already. We easily can observe Google's lack of "platform"; whether or not this is an issue for the company is open for debate. It is not as if Steve revealed some super-secret metrics which reflect negatively on the company.

I live in Hudson, so public transit will never help me much. I used to live in University Heights, a suburb which grew up after the RTA lines were laid out. Ironically, I have been working in Chicago during the week for the past few months and have been getting by a combination of public transit and taxis. $150/mo parking in the condo complex where I am renting coupled with $20/day-ish downtown parking makes a $8 taxi ride a bargain (not to mention the $2.25 bus fare). It also helps that iGoCars (a non-profit zip car competitor) has stationed a car right outside my building.

"missed the last bus in"

This is the problem with small cities' public transit systems. Temporal flexibility is limited compared to driving one's self to work. Need to stay late? You'll be staying all night, or your spouse must come pick you up.

Other issues (observed as a lifelong Clevelander):

1. The elapsed time savings of using public transit is often quite negative (compared to a city like NYC where it probably is often more time-efficient to take the train).

2. Parking costs are relatively low, so there's no tax on bringing your car with you to work. And, you almost never pay for parking at home.

3. You can't live easily without a car, so the potential savings of public transit is reduced to less usage of your current vehicle. You still have to own one, pay for insurance, etc.

4. Density is so low that your first/last mile issues may literally be that far.

All of this creates a death spiral for these public transit systems: Ridership is low, so vehicle frequency is low. This creates higher costs of public transit for people who have more money than time (how much could you have earned while waiting for the bus?), so these systems end-up serving only those with more time than money. This creates a stigma around using public transit which further reduces ridership. Add in white flight and the fear that extending public transit to the "nice areas" will allow "those people" to invade, and it's hard to imagine how public transit could ever work.

This is huge for allowing corps to migrate apps onto Google's cloud. Many open source web frameworks can be used on GAE (albeit with limitations), but de-SQLing an existing app is probably a non-starter for most corporate dev groups. Even if scale-up is an issue past some threshold, the long tail of most mid-sized companies' in-house apps includes a bunch of semi-legacy ones which don't have very high usage levels. Think about stuff like the environmental regulation compliance database -- important, but probably not used by more than 10's of people.

I didn't see any references to ways by which one might use the special JDBC driver to access one's hosted MySQL data. It would be cute to create an app which proxies JDBC calls so that developers can use TOAD/DBViz/whatever to manipulate remote data. Such a driver might also be useful for one-off schema migration tasks. Did I miss something?

The Made to Stick (http://www.heathbrothers.com/madetostick/) authors might say that the surprise of seeing an explicit claim about NOT being accredited would make the FeeFighers site more memorable. When's the last time you saw such a claim?

Not sure that this really counts as an API, but I'd like a converter between things in my life which require polling and a push sort-of interface. Here's an example: I ordered some things from eBay, and I'd like to confirm that they arrived in one piece. After clicking "buy it now", I have to somehow remember to confirm within a couple of weeks that the items arrived. I used to add a calendar task N days out with a note like, "ensure delivery of XYZ". However, I've fallen off the wagon on this of late. The fundamental problem is that there is no ebay-originating tickler event which lets me know that it's time to validate delivery. As an analogy, I think I'd like a way to add cron jobs for my life. Imagine your phone asking you, "Hey, did your package arrive?" If you affirm delivery, nothing more happens. If you say no, a task gets added to your todo list to figure out what went wrong (wife mis-placed package, etc.). You can also say "bug me later."

I'm not sure that I've done a good job articulating what I want, in part because I only have a vague notion of what would help me out. I do know that keeping all sorts of "polling lists" in my head (or even in a calendar/todo list) is very stressful. I'd love to offload the work of remembering this stuff to a trusted assistant.

What other things in life are missing async callbacks?

- Cancelling X service before it auto-renews next month/year/whatever - Generally, any task with the word "confirm" in it - Audit-related tasks such as making sure my client writes me a check for the amount I invoiced.

One could implement basic "tickler" functionality at first without any back-end integrations and then build one-off interfaces for common use cases -- for example, use the eBay API to observe purchases I've made and add ticklers based on the auction's stated delivery dates. Maybe this service also grabs tracking numbers and digs a level deeper for you. "Remember to grab your package from the front porch -- there's a 90% chance of rain tonight!"

Similarly, I was intrigued with the concept of the school independent of the language or country of instruction. This school design is the sort of experience I want for my children.

The article doesn't question the poorer school's choice of measured outcome -- college graduation. Does becoming a college graduate correlate well with a good life outcome, or do the skills required to graduate from college (both academic and character-related) lead to success and/or happiness?

I agree. Imagine not looking at your cell phone and thinking about how some shitty company with a government-granted oligopoly is screwing you over and treating you badly.

The article was un-necessarily snarky. Did the author believe Google to be lying about a human being in control? If not, why make fun of Google for blaming driver error for the accident?

There's so much wrong with this article. First of all, two ideas are artificially intertwined: (1) paying for an app server and (2) using an app server. JBoss is open source just like Tomcat. You can pay RedHat for JBoss support, and you can pay SpringSource (and, for that matter, others) for Tomcat support. Given how expensive Weblogic and Websphere are coupled with how incredibly good JBoss is, you would be hard-pressed to justify writing checks to IBM or Oracle. If you want support, use JBoss and write a check for 1/10 as much. Their support actually kicks ass.

"Use Spring, and you don’t need an EJB-compliant container."

Huh? Does JdbcTemplate now count as a suitable replacement for a transacted ORM? Perhaps the author is referring to Spring's wrappers around other ORMs like Hibernate?

" Manageability is super important, but all I want to know is: 1) Is the server up; 2) is my end-user performance acceptable; 3) has my fault-tolerant architecture kicked-in to solve either problem?"

The open source version of tomcat his shitty management features. If you want even reasonable manageability, you'll have to pay for SpringSource's TC server or something similar.

"has my fault-tolerant architecture kicked-in"

What fault-tolerant architecture? Tomcat's clustered session replication?

"I highly recommend elastic caching platforms (ECP) since it is a technology that is independent of the application server and super-fast. "

You know who makes a really good open source one? JBoss :) And, it's transactional! But, you need a transaction manager if you want to do 2PC with your DB. And, guess what doesn't come with a TM? That's right, Tomcat.

"I question even the need for a container. Seriously, why can’t Java web applications just run on the operating system like the containerless Microsoft .NET applications? "

[Note that I don't know a darned thing about .NET, so I'm speculating a bit] Ummmm...in this case, isn't Windows acting as the container? So, if I create a Linux distro with Java/JBoss included, does this satisfy his need?

"The next generation is elastic application platforms (EAP) that are containerless."

What's up with this obsession about "containerless" platforms? Isn't this like saying, "I want a platform, but I don't want it to be a platform"? The whole idea of a platform/container/framework is that someone else handles a bunch of cross-cutting concerns for you so that you can concentrate on the business-y aspects of your app. Servlets are much more convenient than binding to a socket and serving up HTTP by processing incoming byte streams. Furthermore, how can any of these in-the-cloud offerings be containerless?

Thanks. We've also considered SQLite. One Java wrapper impl (all are 3rd party) makes it easy to pass a direct byte buffer in as a persistence store. So, one could keep a map of database blobs and apply the DB to them as necessary. If H2 is fast enough, I'll take the all-Java solution so that we don't have to worry about native code issues between environments.

I'll have to think about how much I need actual SQL support. It sure does make a couple of my use cases easy (data aggregation), but it may come at a performance/memory footprint cost.

It seems that there are genres of problems for which much better performance and/or memory efficiency may be achieved by avoiding Java's object model and garbage collection. In my other comment in this thread, I mentioned my use case for in-memory database-y stuff. What else? Terracotta sells its "Big Memory" plug-in for EHCache which is basically a ByteBuffer-based in memory cache designed to reduce the number of objects visible to the garbage collector. Certainly manipulation of binary media formats qualifies (and the nearly-abandoned Sun/Oracle-provided libraries for doing so use such approaches, IIRC). What else?

Right now, I'm dealing with a problem where, from an external system, we retrieve up to 1M rows of data -- about 30 columns each (short strings, numbers, etc.). Because the external system can never guarantee that it will provide the same answer to a query twice and because our users still expect to be able to sift through a particular query result without changes to the underlying data, we must store these query results "on our side."

Our app is Java. Even in "short pointer" mode, 30M references to the same string would eat up 4 bytes * 30M = 120M of memory! And, the garbage collector would be displeased because it frequently would have to sift through mature generation regions. So, we are left with disk-based options (a traditional RDBMS included in that mix) or some variation of structs in byte arrays/buffers. [I should mention that we only must provide access to these query results for an hour or so.] While I have found some reasonable flyweight pattern-based schemes for representing structured data in large byte arrays (Javolution has a nice one), I have not found any good data structure libraries which use byte arrays as backing stores. Because the data does not change once queried and inserted into our storage scheme, it seems reasonable to use some look-up schemes (hashes, etc.) which could be persisted in a manner similar to the raw data itself. An additional level of abstraction would be nice, but I'll settle for not writing my own hash implementations.

The route I'm taking now is to use the H2 embeddable, all-Java DB. It has pluggable backing stores, so I can make it pretend that direct byte buffers are disk-like. I don't really need write concurrency, transactionality, and all the other stuff "real" RDBMS systems provide, but this seems like a good trade-off compared to rolling my own in-memory sort-of DB. It may turn out to be fast enough to just run H2 in a disk-based mode.

Does anyone have any thoughts on this?

If we presume that the TSA wants to maximize actual passenger safety given certain budgetary and passenger annoyance constraints, most of its policies and procedures make little sense. When we add in the goal of making the public perceive that it is increasing safety levels and the resulting "security theater" tactics, they start to appear more rational. Does the general public believe that terrorists can't acquire fake ids and print out boarding passes to match? I suspect the majority of Americans don't think they are more secure because of id checking. So, why does the TSA do it? Perhaps the airlines have requested it as a way of circumventing ticket sharing, exchanges, etc.? Have they enlisted the government as the enforcer of their price discrimination model? When things don't make sense, I always presume there are some facts/circumstances of which I am unaware.

Art & Copy (http://www.artandcopyfilm.com/). It's the story of how advertising went from a staid, "old boys" industry to one where creative talent prevailed.

So, is it known for sure that Amazon did not cut him a special deal? It might be rational to incent a "hot" author to offer up a good deal on an exclusive in the Kindle marketplace. Perhaps he got to keep a much higher percentage?