Case-Shiller index is fatally flawed
Thursday, May 31, 2007 at 07:09PM [August 21, 2007: I have changed my mind somewhat on the Case-Shiller index. Despite all its problems, namely that it lags the price drops by 1 - 2 years, it is better for measuring regional price changes than the median price or the OFHEO index. If you want to measure price changes, you've got to look at each zip code or neighborhood's price activity.
So now that the Case-Shiller index finally is showing the price drops, it is okay to use it as a rough estimate of where prices are headed. However, this is important. In San Diego, some micro markets are still rising and getting multiple offers, while most condos are down 30% or more, giving a Case-Shiller index that is down 7% from the peak. Looking at that one number we can never explain the real estate market for a city - it is merely a composite.]
If you're wondering what's going on with housing prices from the peak, never look at the Case-Shiller house price index.
The Case-Shiller index does not measure the change in house prices from last year or last month or the peak of the market, but from the last time that group of homes sold. As such, it's more like a "sold houses index", instead of a housing index. And it lags by 4 months: on May 29 they released the March sales, which were January offers.
That's why the Miami index is up 1%, even though house prices are down in the double digits from the peak. In aggregate, the homes which sold in March fetched 1% more than they did at their prior sale. Again, it doesn't tell us how much the prices fell off the peak, only how much they gained or fell since their last sale.
Thanks to Mike Shedlock for exposing the reasons behind the fatally flawed Case-Shiller Index. I've been an outspoken critic of the Case-Shiller house price index for a long time, because it falsely showed house prices rising for 18 months after they actually were falling.
The San Diego market slowed in the spring of 2004, when condo prices started to fall in the face of doubling inventory (from 3K in April to 6K in June to 9K in September) and falling sales. Despite falling prices, the Case-Shiller index kept rising for another year. It peaked in September 2006 in the Los Angeles/Orange County area, and in June 2006 in the San Diego County area.
At its core, the problem is this: the index looks at sales prices of single family homes compared to the last time they sold, without any consideration whether that house had fallen in between those sales. Say a house sold for $200K in 1999. It tripled in value and would have sold for $600K in 2006. Then in 2007 it sold for $500K. We have a drop of $100K in price, but a rise of $300K since the prior sale. Now, which number should be used in the index? The $100K loss from the peak or the $300K gain since the last sale. The index uses the gain. Voila: the house rose in price by $300K, so we get one more data point for rising prices.
There are more problems. I already discussed that the index is way off on the prices of homes that sold. What about the homes that didn't sell? Those not listed for sale, or those languishing on the market? They're not included in the index at all! So the index is overstating prices, because it omits the houses that didn't sell.... those that lost the beauty contest. So right now, any home on a busy street, or with a cracked foundation, or with peeling paint, is not selling and is not being fed into the index, keeping the index artificially high. The Case-Shiller index measures only the aggregate resale prices of single family homes that won the beauty contest. As I keep harping, aggregate numbers are useless if you want to know the value of a particular home.
As long as we have a bunch of homes selling for higher prices than the last sale, the index will go up. We're starting to have more short sales, and they are finally having an impact on the index. Every time a house sells for less than its previous price, the index registers a price drop. Every time a house sells for more than its previous sales price, the index registers a price gain. As long as the gains outnumber the losses, the index is upward biased. At least that's how I think it works.
There's more.... you know all those remodels done in this housing boom, that have made the houses more valuable? They are totally ignored. The index has no clue about all the $50K - $100K kitchen remodels, backyard barbecue installations, and home enlargements. I have several friends who added 20 feet of living space to the perimeter of their entire homes, because it was cheaper to build on that to buy up and pay all those extra property taxes. Almost every one of my friends has a new kitchen. The index doesn't know it's going up only because Mr. Smith added a $250K remodel. It only sees the increased value of the home.
New homes sales are ignored. All those builder discounts, those big incentives are ignored. Oh, and seller incentives on resales homes, such as paying seller closing costs or agreeing to pay more of the requested repairs are not counted either.
Mish writes about the Case-Shiller and OFHEO (Office of Federal Housing Enterprise Oversight) indices:
... neither model looks at incentives and incentives have been massive lately (but nonexistent at the blowoff peak in 2005).
The differences between Shiller and the HPI seem to be that Shiller does not look at refinancings but the HPI does, and the HPI has a price cutoff based on Fannie Mae conforming loans at $417,000 but Shiller does not. Otherwise they both seem to have a fatal flaw in assuming that there was an increase in price over a 5 year period (on a house that last sold in 2002) when in fact there may have been huge unrealized gains 3 years and a huge unrealized loss in year 5.
The OFHEO Index covers major metropolitan areas, and I use it for my long-range charts, since the data goes back to 1975. Since the index measures repeat sales of the same house, it doesn't have the selection bias of the median data. However, there are other problems with it.
First, the remodeling boom is not reflected, so homes with $100K kitchen improvements which obviously are worth more are now selling for more money, making it appear that the same house increased in value and raising the index.
Second, cash-out refis, which are 88% of loans to Fannie Mae last year, seem to predominate in areas with more appreciation, causing an upward bias in the index. OFHEO came out with a purchase-only index for the US as a whole, and that is 200 bps less.
Third, the lag in the index is 5-7 months. The index records closing for the prior quarter. For example, the offers made in February - May, closed in April - June (Q2), and were released on 9/5/06. So offers made in the end of February - end of May were reported in the OFHEO Q2 index in September. Quite a lag.
What is a better measure of price changes? Better would be an appraisal methodology, where the same group of homes is evaluated by an appraiser every quarter.
Schahrzad Berkland
The Case-Shiller index works fine when prices are rising. It fails only when prices fall. However, the index creators are well aware that prices fall in some regions in 15 year intervals, so the question remains why they do not adjust the index for accuracy. Shiller has been an outspoken person about the bubble, so how did he plan to measure price drops with that index? He knew all along the index would not capture the price drops. I sure hope he grants me an interview to discuss this further.
Reader Comments (13)
Flawed? Yes. Fatally? No. It is what it is. C-S not tracking failed offers for sale in no way affects its accuracy or value. The flaw is in hedonics where improvemnts are not tracked. On the flip side decay of condition is not considered either. Nor are zoning changes. Imperfect does not equal fatally flawed.
I hear people saying that my house is worth this or that, but their opinion is based on comps. A house is only worth what it sells for in the market. That sale price also must go through your suggested appraiser. As such, the appraiser is taken into consideration in all indices as a bank will not sign off without one.
If a house supposed value rises and falls in between, that is of no concern to the house buyer or seller unless it is on the market. Only those on the market can be sold and only those that are sold can be measured. Additionally, all new homes with incentives, phase rollouts and price cuts are ignored for good reason. The problem of comparing apples to oranges, how do you include data from a single purchase with matched pair data? Remember that all the matched pair data is weighted by time interval. There is no interval for a new home. A methodology nightmare that would invalidate your data.
True, the value of particular home is impossible to discern from the Case-Shiller index. That's also true of any and all aggregate data. NOTHING can be said about an individual.
You comment that the prices for condos in San Diego peaked way before the homes did in San Diego. The CS Index never claimed to pay attentio to condos, multifamily, or land. It only accounts for SFR. Did the price of SFR continue to rise till 2006, likely yes. Condos are typically first time buyers who fuel the step up for others. As condo demand wanes and time marches on, SFR prices follow suit.
An index is useful if it supports 3 tenets. It is precise, accurate and valid. Precision refers to how close to the "real" number the index number is. If given multiple similar situations, the index is + or - is 20%, then who really cares about the number. If given the "true" number may times over, the index can't seem to get close, then who really cares. If the model is both precise and accurate, but instead of measuring the price of SFR in 20 major cities actually measures, e.g. the collusion of the NAR and the appraisers, then it isn't really valid.
George Box, a famous statistician among statisticians, once said, "All models are wrong, but some are useful". I would agree that the CS index lags by 4 months. I would contend that it is accurate and precise as it uses only the agreed upon price of a completed transaction, i.e. the true market price. I would also contend that is it is valid as it claims only to be the House index not a Real estate index. So is it useful?
Depends on whether you 4 months is an eternity to you or not...
What's the purpose of the Case-Shiller index? I thought it was to measure changes in values of homes. In a rising market, change in value equals change in price. In a falling market, value can fall but price can be up since the last sale.
RE: condo vs. SFR. SFR peaked in summer 05. C-S went up for one full year afterward.
People need to know what the index measures. It does not measure the price changes of homes. It only measures the price change of homes that managed to sell, compared to their last sale.
We have an aggregate measure of price changes since last sale.
Nothing more.
If everyone's house went from $200K to $800K, and then fell back down to $400K, C-S would show a 50% jump in prices. Fine, if you bought at $200K. Fatally flawed if you happened to be the buyer at $800K.
S.B.,
I think you are expecting too much from a datum.
I thought [C-S] was to measure changes in values of homes.
No, same house sales over time. Value is a much more complex metric.
In a rising market, change in value equals change in price.
All other things being equal. But even that is not correct. Interest rate impacts for instance are not directly tied to value.
What about the homes that didn't sell? Those not listed for sale, or those languishing on the market? They're not included in the index at all! So the index is overstating prices, because it omits the houses that didn't sell.... those that lost the beauty contest.
Absolutely. That's why I like to use an asking price tracker such as housingtracker.net to make up for this particular deficiency.
[Link]
Now if someone would like to track the bids and do the statistical dirty-work of breaking up actual sales into 25th, 50th, and 75th percentiles, then I think that the sales lists from melissadata.com would be excellent source material for this.
[Link]
...
In a falling market, I would expect to see the 'bids' come in below the 'asks'.
I'll ask Bob Casagrand for a percentile chart...he probably has one. In the meantime, here are charts grouping Poway homes by square foot
I'm not sure what Melissa Tracker tell us....the homes left on the market are not selling because they're mis-priced.
Robert, I am now learning what the CS index measures. I think most people expect it to measure the change in prices. The media sure uses it that way, i.e. "housing prices fell x% according to the CS index...." So everyone is misunderstanding what it can do. So there is no method that measures changes in home prices.
Next time you're on Klinge's blog,have him tell you what really happened to those superior properties in the 1980's and the 1990's downturns. Or are you afraid he'll kick you off his blog too? Or maybe you already know, in which case I would love to hear your experience with this. I came here in 1999.
Case and Shiller have PhDs, I don't and neither does the media. I'd be suprised if the media correctly interprets half the published statistics.
Besides, the attention span of the media is so short that they report %up this quarter, %up the next quarter and %down the next quarter. That's all fine and dandy, but over time those percentages mean something different than just a specific quarter. Alas, that's not the goal of the media, its goal is to sell something that grabs attention so they can sell advertising.
As for "In a rising market, change in value equals change in price. In a falling market, value can fall but price can be up since the last sale."...
That's a dynamic that has occured due to the credit market not due to the housing market. Sellers sell based on price of the house, buyers qualify and buy based on mortage payment. The two are not playing from the same deck. Mortage bankers and securitized loans (loans sold as bundles to investors) have "allowed" the buyer to qualify for more money. The resulting 'more money' has bid up the price of the home. *** The value of the home did not inherently increase, but the price did ***
You and I might see plantation shutters, marble slab counter-tops, true hardwood floors (not laminate), developed landscaping, and of course location, location, location as true value. The buyer who qualifies for 700K on a interest-only ARM sees a 'cheap house' at 500K.
As far as "So there is no method that measures changes in home prices." Yes, sort of - which is of course the economist answer. =\
The Case Shiller index measure only SFR resales - as we've talked before - lagged 4 months from publication in 10 and 20 major cities. The real estate market is greater than SFR resales, but Case and Shiller are methodologists. In other words, they are concerned with getting it right even if it's only 25% of the puzzle. (FYI, I have no idea what the actual percentage of the puzzle SFR resales are and I'm guessing it fluctuates.)
So, yes Schahrzad, there is no method that measures home prices. But then again, home is where the heart is. =)
Seriously, Case and Shiller leave it up to us to figure out where their part of the puzzle fits. Maybe, they'll figure out a way to measure all real estate purchases, but until then they choose to be accurate consistently with a part of the puzzle. =(
Melissa data tells us actual sales (i.e. 'the bid'). These are the properties that actually sold; i.e. were priced right and found a buyer.
Housing tracker tracks the prices of properties that are still languishing on the market (i.e. 'the ask').
Each property that sold had a sales price that was recorded in County Records.
Given a sufficiently large sample size, such as all sales for all San Diego zipcodes, one should be able to construct a price distribution chart with numeric bins; i.e. a number of houses sold from $200-$250K, another number of houses sold for $250K-$300K, and so forth.
Hopefully the distribution chart will look like a bell curve in which the 25th, 50th, and 75th percentiles can be computed.
Once these numbers are computed, then they can be compared to the asking price numbers in housingtracker.net.
The difference is then called the bid/ask spread. By doing this, the prices of properties can be looked at in the same way as one looks at the bid/ask spread for common stocks.
My thinking is that this will be a better measure/description of the real estate market than what can be done with the Case-Shiller index. But I could be wrong. It may turn out that this alternate approach is also comparable to what the Case-Shiller index says about a particular real estate market.
We won't know the answer to this, until someone actually collects the data and computes the appropriate bid price percentiles.
The only thing we do know is that housingtracker.net currently says that the San Diego 50th percentile asking price is down a little over six percent YOY.
And the Case-Shiller index also says that San Diego real estate prices are down six percent YOY, as well.
[Link]
However, if the equivalent data from melissadata.com (i.e. the sold property data from County Records) is significantly lower than what's indicated by Case-Shiller or housingtracker.net, then yes, these two measures of the real estate market would in fact be understating current market weakness.
So you think it's accurate that SFR in San Diego rose through June 2006? Hmmm...where? Certainly not in Poway or anywhere else. The only places still rising are coastal and highly desireable properties, but they make up a very small percentage of the sales.
What I am saying is that both Case-Shiller and housingtracker.net are, in all likelihood, understating the price declines for San Diego.
.. And housingtracker.net shows a 9.6% drop in the median 'asking price' from Sept 2005 to Sept 2006.
"If everyone's house went from $200K to $800K, and then fell back down to $400K, C-S would show a 50% jump in prices. Fine, if you bought at $200K. Fatally flawed if you happened to be the buyer at $800K."
This is simply not a reasonable critique of the C-S index. The C-S index takes into account when the last sale of a home was, and there are many such sales, so it can trace exactly what the price is in every period. In the example you give, C-S would observe some houses sold in year 1 and year 2 (sold at 200K and 800K respectively), some houses sold in year 1 and year 3 (sold at 200K and 800K respectively), and some houses sold in year 2 and 3 (sold at 800K and 400K respectively).
Putting this data together, the C-S index would show a 400% between year 1 and year 2, and a 50% decrease between year 2 and year 3. This is a completely accurate portrayal of what happened in the market, so how can you complain about it???
Dan
The CS index lags by 4 months? Not quick enough for you? All data lags. Do you expect real estate date to be in real time?? Impossible. 4 months is reasonable.
And to quote from above;
"As long as the gains outnumber the losses, the index is upward biased. At least that's how I think it works."
Did you say, 'I think thats how it works'? You did. You did say that.
And the CS Index doesn't take into account remodels and selling costs? The implications of doing so would be impossible to manage. As long as the index is consistent, that's the main thing.
I find your critique laughable from start to finish.
Where's the Schahrzad Berkland Index? Nowhere, thats where.
The Case-Shiller index is actually the best index available, beating out the horrible median price. Over the years since I wrote this post, I have come to appreciate it more. They key though is not in tracking the index itself, but the annual rate of change!!! As Robert Shiller explained to me in an email a few years ago when I complained about the index, it is the rate of change that shows price changes. And interestingly, Bob and I show our transition points in June 04 and October 08, exactly when CS shows it using the rate of change.
This month, we are launching our professional report series, for institutional clients. There, you will get real-time data on the housing market, and you can use that to trade the CS index, among other things.