Why is 2008 the worst year for banking crises?

Or, How A Bunch of Econ Bloggers Got Fooled By A Poor Graph

Or, Mood Affiliation For My Bias That Most Academic Research Is Low-Value Rent-Seeking Data-Massaging Justification For More Low-Value Rent-Seeking Data-Massaging

Yesterday, Greg Ip blogged at The Economist about a new paper on banking crises. Someone analyzed 147 banking crises from 1970 to 2011 and found that September was an unusually bad month.

No explanation was offered in the paper, but Greg thought it was very depressing and offered some random theories about harvest cycles and fiscal calendars.

Justin Wolfers was intrigued and tweeted a link to it with the question “Why do financial crises always seem to start in September?”

Tyler Cowen blogged about it with the question “Is September the cruelest month?” While he implied that he considered it a “spurious correlation,” he offered no theories about why September was such an anomaly on the graph that every economist on the Internet now seemed to be obsessing about.

Now I’m just an amateur blogger, but my first thought was, Hmm that is too much of an aberration to not have a good explanation. My second thought was, Hmm 1970-2011 is a pretty short time period to be analyzing this stuff. My third thought was, Hmm September 2008 was a pretty bad year for financial type things… I wonder if that has anything to do with it.

So I scrolled down two pages past that first graph and found a second:

Hmm, kinda looks like 2008 was an abnormally bad year for banking crises! Then I went back to the first graph and noticed that the bars seemed a lot emptier than the second graph. September went up to 25 but nothing else even hit 10. This paper is supposed to be analyzing 147 banking crises, but the numbers on the monthly graph don’t even look like they add up to 40!

Was it possible that this popular graph was unfairly representing September 2008 over a short time period, and not only that, but that two-thirds of the other crises in that time period were missing from the representation entirely? And was this obvious explanation missed by the authors and all the economists spreading this graph and getting concerned about the month of September?

So I followed the links to the paper and found the actual data.

It looks like there are only start months for about 63 of the crises (that’s more than 40, but I haven’t taken the time to see if I counted something wrong or if even some of the monthly data is missing from the graph). Anyway, 22 of the 25 September crises were from September 2008. There was one global crisis that affected a bunch of countries but the authors are counting it as a distinct crisis for each country.

That’s it! Nothing particularly strange about September. No need for silly theories about harvest cycles and fiscal calendars. No need for philosophical musing about how interesting the aberration is. There’s just a big outlier in the very short and very incomplete data!

It would be like analyzing data on Shipwreck Deaths From Giant Boats By Month From 1900-1940 and going, Hmm, I wonder why so many people die from shipwrecks in April? Maybe it has to do with the intersection of seasonal trade routes…

Now I don’t blame the Internet economists for being misled by such a seductive graph… I’ve probably committed worse errors in my years of amateur blogging. I do kinda blame them for not even scrolling down two pages to the second graph that might have helped them at least speculate that maybe 2008 had something to do with it. But mainly I’m just excited to point out an explanation they missed (so please forgive my sarcasm; maybe I’ve been reading too much Sonic Charmer lately).

Maybe it’s because I’m just biased to be more skeptical of academic research. Professional economists seem to think academic papers are very important, perhaps because they write some of them. I think a lot of academic papers are just low-value rent-seeking data-massaging whose only real purpose is to justify more low-value rent-seeking data-massaging. Confirming my bias is page 23 of the PDF: “The data point to several interesting issues that require further research…” Aha!

Maybe that’s normal for “working papers.” I don’t know. But the last thing our economy needs right now is irrational fear about sections of the calendar (“this gives more reason, as if you needed more, to sweat as the fall approaches…” Really, Greg?). I hope others don’t provide so much attention to the next academics who conjure an outlier from some incomplete data.

UPDATE: Thanks to various tweets and links, I’ve seen retractions from most of the people that were spreading the original chart. I’ve been linked by Kevin Drum and Felix Salmon! (The theories in the comments to Drum’s original post are a hoot!)

Also I’m definitely not the only one to have noticed this; there were a couple comments on The Economist post, at least one on Drum’s, and possibly others. I left some initial comments on Tyler’s post as well. But none of them got the attention of the econ bloggers until I blogged about it and tweeted it to some of them. Take from that what you will.

Feldsmarch may be right that I’m being “unfair” to the original paper. Greg may be right that I’m leaping in my opining about the value of academic research. I still think the misleading nature of the graph should have been self-evident to smart academics so close to the data, and either removed or explained, but I guess that’s just my opinion.