Comment by mcherm | Hacker News Reader

Comment by mcherm | Hacker News Reader

I am quite disappointed in the research that went into this article.

(1) Author discovers that Zillow has a specific claim about accuracy: "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home". Author then gripes that this is "a really large margin for error". Really? Because I was prepared to praise Zillow for such honest and clear reporting. It only costs a few hundred dollars per home (Zillow covers nearly every home in the US) to get a professional to produce an assessment, and these assessments are usually only a little more accurate than what Zillow is claiming.

(2) Author pulls up tables from Zillow showing a median error. Which is an excellent way to analyze this and is (I think) extraordinarily small. Author then proceeds to do other stuff.

(3) Author decides to verify the values by checking a total of NINE houses. Is the accuracy of that test expected to impress us? Furthermore, one of these 9 is rejected because "(obviously a data glitch or input error. Dismiss this result)" -- but that's the whole POINT of using median error; it isn't affected by outliers.

(4) Author finds that all of the values (all 3 of them) in Staten Island were pretty far off. Author asks professionals who say that the local market o Staten Island is behaving quite unusually at the moment. Reporting on this is the only thing I think the author did right.

A long time ago I got a real estate appraiser's license. Here is the procedure for appraising real estate, as taught by my state:

1) Pick three homes that you feel are comparable.

2) Pick out the material differences between those homes and your client's

3) Make up a value for each of those differences, and adjust accordingly

(Unofficially speaking) If you don't like the number you come up with, just make different decisions in steps 1-3 until you get the number you want.

In the late 90's I worked for an online mortgage broker, and 9 times out of 10 when we ordered an appraisal, the appraiser would ask us what the selling price was and return an identical figure for their estimate.

I purchased a home couple years ago. The appraiser I used did exactly this.

Amazing that people still pay for that.

"People" don't. It's a required part of the due diligence to get a mortgage. No appraisal, no loan.

For the most part, the banks and brokers re-sell those loans immediately, so they don't stay on their books. All they need to protect them from lawsuits later is proof that they followed the process to the letter -- which involves getting a written appraisal.

No mortgage broker or Realtor ever got rich by picking a fight with the appraiser to lower the appraised value ... there is literally nobody present at the table who has an incentive to push the value down.

Aside from the buyer, who can pay for his own appraisal.

Usually a buyer has already agreed to buy a house at a certain price. So if the appraisal comes back low, the seller doesn't lower the price. Instead the buyer just doesn't get a loan so they don't get to purchase the house they had their heart set on.

A friend of mine saved $15K on a house because the appraisal came in low.

If the bank won't write a loan for the agreed price, the original contract is void, since it's contingent on loan approval. If the seller's heart is set on moving, he might well lower the price.

Really depends. In the Bay Area these days, sellers will usually only take offers with zero contingencies (because they can). Even if you do get a financing contingency in your offer and the deal falls through because of a low appraisal, there's almost always a back-up offer waiting in the wings that can cover the low appraisal with an additional downpayment (the bank doesn't really care about if the house appraises for the selling price, they just want to make sure that their Loan-to-value (LTV) ratio is OK.

That's really playing with fire. Good RE agent would tell their client so. Even if you're rich and buying in cash, at least inspection contingency is a must, but otherwise unless you have tons of money and don't really care for nasty surprises inspection could uncover - zero contingency is the riskiest bet one can make in one's life. My opinion, if you're on the market where sellers won't accept a contingent offer, it's better to get out of that market. I've bought house in BA recently, and there were plenty of sellers willing to take contingent offers (backed by some solid bank paperwork, maybe, but not just requiring to commit blindly). Of course, BA has many different places and in some market could be so hot sellers can get away with anything. But it's usually not good to get in such place as a buyer.

Usually, the seller provides an inspection report performed in advance. It's a big leap (I did this a year ago ...) and my Realtor is as old-school as they come, but if you want a single-family property in a hot area, it seems to be table stakes.

Sure, just depends on whether or not there are other parties who will pay the Delta.

Happened to me when I sold my house the value came in 10k lower (really... now that same place is 30-40k higher than what I was asking a year ago).

I understand that it's required, and that organizations often follow processes long after they're revealed to be foolish.

But it's been a decade now since the housing bubble revealed that appraisals are meaningless.

Why has no one eliminated them?

They are required by law or by long-standing policy in a government-run body (which is practically the same thing). Laws DO change to keep up with the times, but they do so rather slowly. In 50 to 100 years, if there is continuous clear evidence that home assessments are highly unreliable and not particularly profitable to anyone, then you should expect them to be eliminated.

Because they reasons they were required were still valid, so instead efforts were directed at the factors that made them ineffective at addressing the need motivating them.

Whether those efforts were successful or not is another discussion.

This. As long as the appraisal is close enough for plausible deniability, it serves the purpose. The originator just needs a number that's close enough for reality that whoever they sell the loan to (probably the government) can't come back and accuse them of fraud.

Apparently you've never actually taken out a mortgage.

When we were selling our home we didn't have a good idea of what it was worth so we called an appraiser and the first think he asked was "what are you selling it for?" Then we found out that even if we paid him a couple hundred bucks he wouldn't even come out and inspect the house before "appraising" it.

We ended up just guessing high and lowering our price a couple of times until someone bit.

That actually makes some sense. That number is supposed to indicate the maximum someone would pay for the property. If you're in contract, especially after competitive bidding, then we know that number: it's in your contract!

The appraiser just needs to make sure that the number isn't fraudulent or crazy.

As an example of above, A couple of years ago my wife and I were selling our home and the appraisal came back lower than agreed sale price, we asked appraiser to pick different comparables and the appraised price went up 10-20k

Thats interesting, why would the seller have any control over the appraisal? The mortgage broker who requests the appraisal is typically working with the buyer.

I don't think this is "control", the appraiser [sp?] did not have to agree to do that.

You are correct they didnt have to agree to change anything. But there were other houses that were closer ( one of the comparables that he pcked was in a different neighborhood and was waaay cheaper ) and houses that were more similar so we pointed that out. He agreed and redid the estimate with houses that were closer both on distance and in size/amenities. All of this was throught the realtors ours and theirs.

True, but the appraiser has zero incentive to disagree (unless it's overtly fraudulent there's no such thing as wrong appraisal) and probably some incentive (repeat business) to keep the agent they are working with happy.

This is how it USED to be done, and was a significant contributor the housing bubble that lead to the recession.

It's different now, not necessarily less subjective but the incentive for creative appraisals has been drastically reduced. Prices have to be justified and mortgage brokers can't interact with the appraisers now. Apprasing has to be entirely separate from the underwriting process and is now entirely blind. In some states, appraisers are now accountable for significant deviations. The turnaround time is also significantly longer. I've heard reports from some people of appraisals in bigger markets taking as long 9 weeks.

It's actually a pretty interesting process, ripe for disruption if anyone wants to take a crack at it.

The separation part only sounds good. However, if you get an incompetent appraiser, there's literally nothing you can do about it. It's also hard to weed out the incompetent ones, as they are now independent from their customers. We got an incompetent on our last deal. Couldn't find the house. Misidentified the street. Couldn't find the electric panel. I could go on. Took several weeks to get the appraisal done, and there was nothing we could do about it because independent.

ya, i understand. but that sounds like a very different issue.

It's kind of scary that million dollars' decisions (and in aggregate, probably billions) are made on the basis of solely "let's look what others are doing and then fudge it based on gut feeling until it looks kinda plausible" and that's supposed to be expert verdict. Not that I can offer any better idea :)

Haven't you just described 90% of the VC investments made in startups in the last few years?

Well, I guess it depends. Some investors do a lot of due diligence before investing, ask for financials, plans, market analysis, etc.

Yeah people dump on Zillow quite a bit. Probably because home values can be a very emotionally stirring event.

Zillow's aggregate economic data is solid, often a better predictor than the Case Shiller Index. I expect that Zestimates will improve over time as the data trickles down and lessons are learned.

This is correct. I used to work at Redfin and estimates are a sore point for homeowners. It's one large pissing contest for their egos. Everyone has the same complaint, "Why isn't my home priced as high as that one?!?"

I'm the other guy. I bought my house for 260k a year ago, redfin now lists it at 330k, and there is just no possible way that is correct.

I'm in the same boat as you. The Zillow estimate in my neighborhood seem to be about 15% higher than the sale price on any particular property. It's kind of academic for me, since I don't plan on selling any time soon, but it would be a mistake to do financial planning based on those estimates.

The funny thing is the estimate doesn't change when a sale closes. I would think the surest indication of what you can get for a property is what it sold for last week.

> "Why isn't my home priced as high as that one?!?"

Could Redfin not explain why?

I've never bought a home in an area Redfin is in, so I havent used the site before. If Redfin knows the formulas/parameters used for the estimates couldn't they break it down and do side by side comp of $ amounts?

> "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home"

Yes, this sounds pretty good to me. I spent some time trying to do something sophisticated for the estimates on my site https://houseprices.io/ using repeat sales of similar properties weighted by distance, but I wasn't getting anywhere near that. I gave up and just used the Land Registry index for the county in the end. There are just so many potential local factors which you're never going to capture algorithmically, not least of which is how much the current owner might have spent doing it up. Although I do get the occasional irate message in my Contact Us form telling me exactly how much!

I feel like disappointing research is going to be a given when you have such an obvious conflict. The author has a strong motivation to make Zestimates seem bad (the less reliable the resources you have to buy/sell a home on your own are the more you'll need to contact them) and a million degrees of freedom (which neighborhoods in which cities, price floor for consideration, how many examples to pull ...) to get the answer they want.

"Research" of this type should probably never be trusted.

I'm not sure at all how you can make conclusions from just 9 houses. I've been house hunting recently. Even houses in the same neighborhood are very different, and there are a lot of factors going into valuing a house. Also may be very subjective - some people would pay a lot for things that other people don't care about. Maybe Zillow is uniquely bad at one or two out of 20 factors - e.g., it can not detect a house in a good neighborhood that has been neglected and needs a lot of repair (I've seen houses in very good places that looked great from outside and required tens of thousands dollars of work on the inside just to be livable - maybe that's what happened to that -75% house - maybe it had a recent fire or flooding). Or maybe it doesn't value having a pool correctly. Who knows. But it's impossible to figure that out from only 9 houses, 3 per huge market! You need hundreds of data points to smooth out the individual differences and cancel out noise from all different criteria. That's exactly what Zillow's advantage is supposed to be - to aggregate thousands of data points. Maybe they do a good job, maybe not, but I don't think you can evaluate it by just using mere 3 houses per market of millions!

Reply on news.ycombinator.com