ShabbyDoo

Thinking more generally than just insurance, there are a lot of things I buy on a recurring basis which, if I just allow my current providers to have their way with me, tend to get more and more expensive over time. Lawn weed control/fertilizing is the example which comes immediately to mind. I'd love to set-up a profile of my lawn with a site which would then allow providers to quote me prices for services semi-continuously. This would keep my current provider in check.

I spent a few years working for a P&C insurance company. The most valuable variable for determining the expected value of future claims (and therefore one which significantly impacts price) is one's credit score. That Leaky is quoting without knowing my credit score means that the rates provided can't be terribly accurate.

Yet another example of how hard it is to predict how a product/technology will be used or what societal benefits/detriments it might provide.

Wow. That in itself is a great story which should be heard.

Let's say you are trying to get a job with the FBI or obtain a security clearance. You will be asked to provide a list of family, friends, bosses, teachers, etc. from throughout your life, and those people will be interviewed (about their knowledge of you, not their own propensities to get too drunk, etc.). Lots of public and not-so-public records will be analyzed, etc., etc. Not much privacy.

What's interesting is that the purpose of the background check process seems to be threefold (as I gleaned from talking to an FBI Special Agent who interviewed me about a friend):

1. Identify possible ways in which the investigated person might be manipulated. Having an affair? Had an affair a few years ago which the current spouse doesn't know about? Gambling problem? Drug issues? If so, you might be susceptible to blackmail or bribery.

2. Identify propensities for dishonesty or lack of trustworthiness. Did you cheat on tests? Do you follow-thru on promises? I was asked a lot of questions about how much I trusted this friend.

3. Identify possible 3rd party allegiances. Are you someone identified by the mafia at age 15 as a person who will go to college, look sparkling clean, and then infiltrate the FBI? To make sure, the FBI interviews people from many phases of one's life. It would be really hard to have seemingly normal friends while you were secretly off training with some foreign terrorist organization. I suspect I was also background checked, but I never signed-off on it. The FBI seems to be using the social graph transitively to verify that you aren't hiding anything. No proof of this last statement though.

I've been told that, at least for normal governmental security clearances, the goal is not to determine if you are a good or bad person -- it's to determine if you are hiding something. If your wife knows about the affair you had five years ago, it will be hard to use that knowledge to blackmail you. However, if it remained a secret, you might do something not so good for the USA to save your marriage.

I was asked about the additional names (expressed only as initials due to space constraints) on my friend's condo deed. She had purchased the condo while in law school, and her parents co-signed on the loan. Only after the interview did I realize that the FBI was confirming she did not have a sugar daddy!

There is one restriction in particular which I believe has a huge effect on the legal industry's slow pace of change -- only lawyers may own companies which provide legal services to clients (vs. in-house counsel). So, want to start an online, virtual law firm? Either act as a referral service or make sure you never want a non-attorney to have equity in your company. Why does this rule exist? I haven't a clue.

You do already, I think (speaking of The US).

My father, formerly a union rubber worker in Akron, once noted that, when the big tire manufacturers stopped paying workers in cash on payday, their wives were ecstatic. This was because, on payday, a not insignificant fraction of guys would take their wads of cash and head off to the bar. Or, they would take out $20 and hand the rest over to their wives, telling them that the remainder was the amount they had earned. Apparently, it was not until the 1960's that paychecks became the standard in Akron for union laborers at the Big Three. My dad's view, from having talked to many men working in the factory, was that the majority of the era's blue collar married men simply turned their checks over to their wives, asked for a few dollars for beer/cigarettes, and allowed the women to manage the financial matters of the household. This is in stark contrast to anecdotes I've read/heard about the wives of white collar men during the same period who were largely in the dark about household finances.

Perhaps a significant fraction of men were never terribly ambitious, but they maintained social acceptability through the ease by which they could earn a middle-class living as a union factory worker? Their wives seemed content to "wear the pants" by taking care of financial affairs. Of course, to outsiders, they likely allowed the man to maintain some sense of dignity by feigning subservience (my presumption of the era's cultural standards). I'm not suggesting that factory work was not physically taxing, but jobs were generally available for anyone who wanted one. My dad recalls collecting factory job offers from Firestone, Goodyear, and BF Goodrich in a single day. Some guys who couldn't read or write very well would bring someone with them to fill out the application forms.

Is it that today's men lack the ambition of those from a generation before, or has the disappearance of union manufacturing jobs and other for-the-asking means of maintaining a middle-class lifestyle take away the false floor keeping men grounded?

Good point. Returns usually are exception conditions, so one need only factor in a fraction of the "cost" of a return when making most purchases. However, when I order two sizes of the same shirt, I obviously will be returning one of them.

From my wife's perspective, that's an added bonus -- the UPS store is right beside Starbucks :) In all seriousness, the post office, UPS store, FedEx Office (formerly Kinkos), etc. are all located in places we pass by during other common excursions (grocery store, etc.), so the marginal cost is low.

I use monoprice as well for cables, etc. Well under half the cost of Amazon makes it worthwhile.

Couldn't they use the same legal mechanisms (wholly-owned subsidiaries, etc.) employed today to avoid charging sales tax to customers in located in states with Amazon distribution centers?

Amazon probably either explicitly hedges already or does so implicitly by "purchasing" a forward contract for shipping rates from UPS. The risk of varying fuel costs exists no matter who takes ownership of it.

I have been surprised that Amazon has not vertically integrated into delivery and distribution. Imagine if they had bought DHL. They would have turned what must today be a huge variable cost into a largely fixed one. I don't have time to write about all the interesting innovations in distribution which could result from combining warehousing and delivery. Perhaps fears of antitrust lawsuits and hardball tactics from UPS/FedEx keep Amazon from doing what I consider an obvious next step?

"Amazon spends more than $90 a year for each Prime customer, losing $11 annually for each subscriber."

"Losing"? If the other stat of purchase volume increasing from $400 to $900 is true, Amazon is buying volume business for just a 1% discount. I wonder how the profit margins skew between average purchases and those which only happened because a customer had signed-up for prime? I recently ordered some collar stays and had them overnighted for $4. Their cost was $20 for what amounted to some little pieces of stainless steel (exactly what I wanted). The margin on these has to be much better than a DVD player.

"82% of Prime members buy on Amazon even if the item is less expensive somewhere else"

The supposed "source" referenced another study (sans link), so I can't get the answer to the obvious question: Was "less expensive" computed using the marginal cost to the customer of this particular purchase? Meaning, do other sites offer the widget for a few dollars less but charge a lot for 2nd day shipping?

I'm a Prime customer, and I've mostly given up on comparison shopping for small items. Amazon's prices are usually good enough that, especially coupled with the free 2nd day shipping, my effective "wage" for comparison is too low to make it worth it. Nevermind the "it just works" feature of Amazon's system (not just website -- the entire process by which a widget comes to my house).

My wife and I have forced ourselves to change our shopping behaviors and ask ourselves before thinking about leaving the house, "Do I really need this thing now, or can it wait a day?" $4 for overnight shipping is a bargain compared to wasted time, gas, etc. Stores are harder and harder to justify. That my wife finds ordering a bunch of clothing, trying stuff on at home, and returning what she doesn't want to be much more efficient makes me wonder what the future of retail clothing becomes. She sometimes even regards the "home try-on" process as superior because she can see how a new item of clothing looks with other, previously purchased garments.

"They're going to continue building Motorola branded devices and it's going to be the same team doing it."

Too bad. My Droid RAZR suffers from all the crapware added by Motorola and/or Verizon. I'm sick of being asked if I would like to use VZNavigator instead of Google Maps. My personal hope for the acquisition is that someone at Google would tell them to stop making their phones suck so they could squeeze out a few additional pennies from each customer. From Google's standpoint, not shipping a product which maximizes customer happiness is penny-wise and pound foolish.

I live with my wife and our two young children, but from ages 23-29, I lived alone. While living alone (especially sans significant other), I used to think, "Oh, it would be so wonderful to come home to a house full of activity." Now, I am beginning to resent the family cat because its presence (and seemingly continuous interest in being fed, going outside, coming inside, etc.) makes it impossible for me to be without distraction, even if my wife and kids have gone somewhere. The grass is always greener, I guess. I've turned what was originally designed to be a formal living room into my office, complete with locking door and couch. This helps, but it's hard for a two year-old to understand why daddy doesn't want to come out and play.

>IDE's really know what is good for what situation >and just autocomplete whatever is appropriate.

If people claim to be doing something everyday which requires detailed knowledge of X, they ought to be able to tell me about X. An IDE can't know if a List or Set is more appropriate. To know this, it would have to know my intent. I agree that higher-level abstractions (declarative statements implying the need for one over the other, etc.) might be in our future, but the people I'm interviewing claim to be writing Java. I don't expect a Java developer to know any particular processor's instruction set, but I sure would expect, say, a person who recently wrote an x86 assembler to have detailed knowledge in this area.

tl;dr; I only ask detailed questions about technologies of which a candidate claims recent and deep knowledge. Wrong answers suggest stupidity or dishonesty.

Let's say I'm interviewing someone who claims to have been a professional software for a decade and that most of his work was in Java. I know Java quite well, and I know what sorts of things one HAS to know about Java if he doesn't fundamentally suck as a software developer and isn't lying on his resume. If you claim to have written Java code for the past five years but can't explain the difference between a Set and a List, you are either stupid or dishonest. One of my favorite questions for a supposedly experienced Java developer is to show them a code snippet with a try/catch block and ask them to trace through the code's execution in both the case where an exception was thrown and the "happy path". Roughly 40% of the people I've interviewed who have claimed years of recent Java experience (like they were coding in Java the day before) think that a method's execution terminates when a catch block is entered and that block's code finishes executing! How could one possibly write correct code without understand this? How much evidence to the contrary must they have ignored? One developer complained to me that my question was hard because he was "mostly used to dealing with SqlException and not other exception types." He then complained that he was an experienced Java developer and didn't see how my question was relevant to the job for which he was interviewing. Want him on your team?

I recently worked on a project where, to cut down on space, I built a custom "compressor" for lists of tree-like records. You might think of a Person record with N past addresses although this was not the actual domain. No records were cross-linked (at least not formally in the code) and the records were trees, not more general DAGs. The data contained a lot of enumerated data types (address type, gender, etc.). I didn't really care about the space usage for 1K records, but I cared about 1M. I used variable length encoding (a hacked-up subset of the Apache Avro project) for integers to take advantage of most of them being small in the datasets. Lots of lookup tables for enumerated values and commonly-repeated in practice string values. Implicit "keying" based on a record schema to avoid specifying key identifiers all over (our data was not very sparse, so this beat out a serialized pattern of KVKVKV etc.). I thought about taking advantage of most strings having very limited character sets and doing Huffman encoding per data element type, but the results were good enough before I got there. A co-worker also noted that, because parts of these records were de-normalized data, subtree pattern recognition could provide huge gains. I added some record length prefixes to allow individual records to be read one-at-a-time so that the entire dataset would not have to be read into memory at once. IIRC, compression speed was 2-3x gzip(9), and large record sets were 1/10th the size of using Java serialization plus gzip. [Yes, Java serialization is likely the worst way to serialize this sort of data]

Was all of this worth it? It solved the problem of not burning through network and memory, but it was a local optima. The root problem was that this data came from another system which did not provide repeatable reads, and providing them would have been a massive effort. However, our users wanted to meander through a consistent data set over the course of an hour or so. To provide this ability to browse, we throw these records into a somewhat transient embedded H2 DB instance. The serialized format is required primarily to provide high availability via a clustered cache. In retrospect, I would have pushed for using a MongoDB-esque cluster which could have replaced both H2 (query-ability) and the need for the the serialized format (HA).

It surprised me that there were no open source projects (at least Java-friendly ones) which provided compression schemes taking advantage of the combination of well-defined record schema and redundant-in-practice data. Kyro (http://code.google.com/p/kryo) comes closest as a space-efficient serializer, but it treats each record individually. Protobufs, Thrift, Avro, etc. are designed for RPC/Messaging wire formats and, as an explicit design decision (at least in the protobufs case) optimize on speed and the size of an individual record vs. the size of many records. The binary standards for JSON and XML beat the hell out of their textual equivalents, but they don't have any tricks which optimize on patterns in the repeated record structures.

Is this just an odd use case? Does anyone else have a similar need?

Fascinating that you've actually done this! What is sad to me is how little effort was required to bring about this level of transparency. I'm not suggesting that you did not work hard on this series, but it's not like 10 investigators spent a year interviewing everyone possibly connected. Perhaps the bright side is that you've demonstrated the potential upside of non-profit transparency work by showing how low-hanging the fruit might be.

Interesting point. I wonder what graph of oversight level vs. corruption level looks like. I'm sure the point of diminishing returns occurs at only 1/10th a dedicated journalist. However, the marginal societal value (cost of corruption compared to cost of oversight) might continue to be positive for awhile as investigators are added.

I have wondered for some time what the effect would be of the following transparency experiment:

With an experienced investigative journalist as a mentor, "deploy" a handful of young journalists to report via a blog-esque medium on the politics of some randomly chosen small city. What would happen if every police incident was investigated to find out if the powerful were granted special treatment? If every claim made in a city council meeting was verified? If every motive behind every vote was examined? How about tracing down budget expenditures to see who was scheming to defraud taxpayers? Would the city's citizens care enough to even read the blog? Would this transparency change government for the better? It certainly would give Americans a baseline for the level of corruption and inefficiency in local governments.

" But in future, the university says, there will be mechanisms for checking identity and verifying work."

This increases the value of the certificate as proof of knowledge and competence. Let's imagine that MIT offered 20 engineering courses in this format. You go through the effort of honestly taking these courses and add them to your resume. A potential employer says:

"Well, these courses are really rigorous -- much more so than those for-profit online 'universities'. While MIT will allow me to verify that a single online account holder claiming to be this person completed these courses, there's no proof that you didn't cheat or pay people to take the tests for you. Just look at how many CS homework projects are listed on Rent-a-Coder! I have to bring this person into my office and check them out myself to be sure."

I have argued previously that this "checking out" process isn't so expensive that an impressive resume would be passed over, and I still think this. However, what if a student had the option of paying a fee to take a test at an approved testing center which would check identification, take a photo of the student (available on MIT's site along with the course certificates), and ensure that this person took the test himself? Those taking actuarial exams and such already take tests at such locations. For those who didn't want to pay, lived somewhere without such centers, etc., certificates could be given which would have a lower level of authentication. If you are taking the time to complete a course and want to prove to others that you did not cheat, wouldn't it be worth paying an extra $100?

Do you list your degree on your resume? Do those approaching you with proposals for work know that you went to Stanford? I agree with your view that a degree itself isn't inherently valuable, but I wonder if you are able to measure its benefits so easily.

Yeah. Done historically by stockbrokers making cold calls. Every week, "Now, don't put your money with me just yet. Here's what I'm putting my clients into this week. Just watch Stock XYZ this week to see what happens." If you pick three highly volatile stocks every week, make a lot of calls, and drop the prospects to whom you have given "bad" advice, you'll end-up with a decent sized set of potential clients who think you are a god.

>Why not just take more profit?

Why do businesses not just raise prices on everything? Unless they are colluding and very good at limiting competition/alternatives, a competitor will happily undercut them.

I'm happy that airlines are beginning to decouple the service offerings of a seat, food, and baggage handling. This means that I can choose a flight which maximizes my happiness at a particular price point. Is United offering cheap fares but charging for baggage? Maybe I'll choose them when making an overnight trip. No food service? My local gourmet grocery sells very good take-out lunch boxes, so I'll just bring one of those onboard.

So, charging by weight is just another means of pricing innovation. What are some other ways the airlines could achieve price discrimination? The days of charging extra without a Saturday night stay seem all but gone thanks to Southwest and other innovative airlines.

This sort of thing could set-off an arms race between police and those who want to record them. However, the police today seem to stifle recording in a grass roots, individual way. Would the police chief be embarrassed to explain in a room full of journalists that his department "jams" legal communications equipment as a matter of practice?

Imagine a GoPro-style camera which transmitted wirelessly to a nearby smartphone. From there, the receiving app would sync it to a server ASAP. If you knew you were entering into a situation where video you were capturing might be confiscated, you could tape the phone to your thigh with a lot of duct tape. By the time the police organized themselves sufficiently to cut the tape off of your body, the important video likely would already be uploaded. Could this scheme be expanded upon by some sort of steel lock box around one's waist/ankle/etc? That would give the phone a lot of time to do its work. Furthermore, the box could contain an additional microphone which would continue recording after initial camera seizure. In cases where groups of protesters feared arrest, could Bluetooth be used to create a Tor-esque upload-to-the-cloud network which would eventually reconstitute everyone's video? I'm imagining a bunch of phones talking to each other in a mesh and trading blobs which could be reassembled server-side.