State of the economy: when are perceptions reflecting objective economic performance?

Reading Time:

People's perceptions of the economy in the U.S. have improved, but the share of people who are "satisfied with the way things are going" is low and no greater than in 2009 or 2010, as Ben Bernanke recently pointed out. Claudia Sahm doubts that "[we should let] economic policy off the hook" and shows that respondents' disapproval of government economic policy has increased sharply since late 1990s.[1]

Sahm's and Bernanke's articles, and the common narrative that "even if the economy is growing, most people don't feel like their lives are getting better" prompt me to ask a couple of questions:

  • What outcomes do citizens view as most desirable? (Research shows that unemployment lowers well-being more than inflation, but what about other economic variables?)
  • How accurate are people's perceptions about the state of the economy?
  • Are there clusters of countries where citizens view their national economy similarly? And what observed outcomes do these countries share?

Already ten years ago my boss Adam Posen predicted that we were entering "an age where economic performance will not be a dependable predictor of election results". That sounds painfully right: most advanced economies are growing, and unemployment has fallen a lot since 2009, but voters in all directions are in a mood to "fire the coach" as if "things could not get any worse".

To make sense of this, I took survey data which reports the proportion of people who believe the economic situation in their country is good (Eurobarometer reports this type of data for 32 European economies). It turns out that the subjective assessment of the economy correlates quite closely with the objective variables that journalists and policymakers care and write about (GDP growth and unemployment).

This doesn't mean that we measure all the right things (or that prosperity is widely shared as we might hope), but knowing the precision of perceptions is still a big deal. If people believed lies about deficits, the unemployment rate, or whether the economy is in a recession, then good luck holding leaders accountable...

Perceptions vs. reality

A quick look at unemployment: in countries with high unemployment, fewer people say that the economic situation in their country is OK.

Generally, fewer than one fifth of people see the economy positively when the unemployment rate exceeds 10%, though there are some exceptions:

The correlation between people's subjective assessment of the economy and real GDP growth is positive: in countries with fast growth, people do also tend to perceive the economy is stronger. But the statistical relationship between last year's GDP growth and contemporaneous assessment of the economy is rather noisy:

Source: Eurobarometer sentiment data (2015); Economic data from the World Bank is for the preceding year (2014), hence "lagged".

OK, so subjective perceptions are correlated with hard economic indicators. This suggests that GDP continues to be useful measure (though it is an imperfect metric in many ways, as documented by many scholars and institutions).

Modeling perceptions

Ask any economist to predict a variable (subjective assessment of the economy, in this case) and he or she will give you an OLS table. If you ask for something better, you are bound to get a more involved table with more regressions, enriched with indicator variables, interaction terms, nonlinear transformations, etc. If the research note or paper is formally reviewed, someone will ask for "robustness checks", basically asking the author to prove that when new variables are added to the model, the reassuring superscript-star doesn't vanish ("the effect remains significant").

Like Andrew Gelman and others, I am unhappy about some parts of that process, because I have seen the concept of "significance" used too poorly (or dishonestly) a bunch of times, and my immunity system eventually started to bark at me... But let's see some regression anyway:

Don't worry, we can do a lot better. My goals for modeling perceptions are a matter of personal taste, of course, so let me list what I'd like to achieve. In an ideal world, I would hope to:

  1. Use the information contained in GDP, even though it happens to be statistically insignificant in regression (3).
  2. Make more accurate predictions about sentiment if I only had objective indicators about the recent state of the economy (look for a way to decrease mean squared error).
  3. Produce a model that is easily interpretable.
  4. [Desirable but not required] Show you a model that a non-economist would understand.
  5. [Desirable but always hard] Aim for good out-of-sample (OOS) performance.

Lots of tradeoffs are involved. I will get kudos on objectives #3 and #4 if the model is just a rule of thumb (say, predict Y=20% if the unemployment rate is "high" (but how high?) and predict Y=60% if the unemployment rate is low) but I will fail on #2. I can improve #2 by estimating a regression with quadratic predictor:
$$ Economy_{it} = \alpha + \beta_1 GDP_{i,t-1} + \beta_2 Unemp_{i,t-1} + \beta_3 Unemp_{i,t-1}^2 + \epsilon_i $$

but I will do a lot worse on objectives #3 and #5 (R-squared would increase from 0.4 to nearly 0.6 but then for countries with unemployment surpassing an inflection point, the partial effect, statistically speaking, of unemployment on opinions about the economy would be positive, i.e. wrong![2]).

Model alternative: A decision tree

A nice, parsimonious alternative to a series of regressions, is a machine-learning method called CART (Classification and regression trees). I have 32 countries, some economic variables, and I tell R to classify the countries into four groups, while minimizing mistakes:
\min dev(\bar{y}_G) = \sum_{G1} (y_i - \bar{y}_{TOP})^2 + \sum_{G2} (y_i - \bar{y}_{2nd})^2 + \sum_{G3} (y_i - \bar{y}_{3rd})^2 + \sum_{G4} (y_i - \bar{y}_{BOTTOM})^2

The decision algorithm for classifying countries into four groups that maximizes accuracy while maintaining simplicity looks like this:

(We could ask for more groups, but then we would be throwing goal #5 out of the window).

The tree tells us that:

  • As long as the unemployment rate is low enough, there is no need to know a country's GDP growth; we can make accurate predictions about perceptions knowing only the state of the labor market.
  • For the "relatively low unemployment countries" it is informative to do another split and separate low- vs. medium-unemployment places.
  • When the unemployment rate is high then it is statistically optimal to check the growth rate of the GDP. When jobs are scarce, faster GDP growth mitigates the pain of high unemployment.

How is this model doing with respect to the five earlier goals?

Goal 1: Use all available data


Goal 2: Model accuracy (do we have a better fit?)

You might be suspicious at first: if the tree predicts the same value for a several (7 to 10) countries at a time, can it possibly by be accurate? Well, let's look at the first group of countries (those where sentiment is highest, so the tree predicts 67.9% for each of these countries) - I'd say that the tree is doing a good job in four cases:

In some instances, the model is not doing a good job: it is underestimating economic sentiment in Germany and Denmark, and overstating it in the Czech Republic.[3]

But how do these errors compare to the linear model (regression 3 above)? Here's the answer when we show both models - in the majority of cases, the green dot is further from the actual value, meaning that the OLS regression makes larger mistakes:

The linear regression does a horrible job predicting sentiment in Germany, Denmark, and the Netherlands. The tree did not do a great job in the case of Germany and the Czech Republic, the but regression is a lot worse overall...

What about the other country groups? Instead of going through actual and predicted sentiment for all countries, let's look at a boxplot of squared residuals for all groups:

It turns out that the tree model does extremely well for those 17 countries which are either in group 3 or 4. The median squared residual is lower (or same, in the case of group 2) if you use tree model, and the dispersion of residuals is always larger when the linear model is used to generate predictions.

The boxplot suggests that one would benefit the most from switching from LM to a tree when assessing sentiment in countries with high employment and fairly fast growth (well, relatively speaking).

Overall, the residual mean deviance of the best of the three linear regressions was 0.036 (for the whole sample). Using the tree, we get Dev = 0.5306 / 28 = 0.01895, a nice improvement.[4] The residual standard error also improves: it falls from 0.199 to 0.138.

If you just kept guessing the mean value, namely 34.6%, then your RSE would be 0.35 (and your MSE would be 0.062).

Here is a comparison of the size of squared residuals for each of the three approaches:

To make this exercise complete, the countries and the groups into which they were classified are listed here:

(Below is the output from R; I preferred showing the tree using Omnigraffle.)

Goal 3: Interpretability

If you know two data points about a country, you can immediately tell which of the four groups it falls into. Classification is immediate. No spreadsheets or calculators are involved.

If you have the same information about a country's economy, and you only have the table of regressions, you will be able to tell me what is the share of people who see the state of the economy as good, but it will take a moment.

Goal 4: How much background is needed?

Reading the table requires some past experience with social sciences, or a class in statistics. The tree can be followed by anyone.

Goal 5: Have we relied too much on in-sample parameters?

I'm happy to stick with the tree, but have no idea how the model will perform out of sample, when I plug in new economic data into the tree in the future. (We just don't know how people's expectations will change, and whether people will value the same outcomes equally. I used the latest sentiment data for this exercise, but could have used historical data to train the model, and do proper OOS tests; maybe some other time.)

Where next

It would be useful to know how opinions about the state of the economy correlate with perceptions of government competence (or trust/credibility and related concepts).

It is also surely worth exploring how much perceptions are shaped by distributional issues. GDP is an "average income" concept, but median (disposable) income is bound to be a good signal about the state of the economy as well.

According to a new MGI study "[b]etween 65 and 70 percent of households in 25 advanced economies ... were in segments of the income distribution whose real market incomes ... were flat or had fallen in 2014 compared with 2005." One of the conclusion of the study is that "[t]oday's younger generation is at risk of ending up poorer than their parents". Such trends may have an impact on how people view the economy. I plan to look at these issues in future posts.


Data: Top 5 and bottom 5 countries

Country Economic situation is good (percent) GDP growth (2014) Unemployment
Germany 86 1.6 5.0
Denmark 83 1.1 6.6
Luxembourg 79 4.1 6.1
Sweden 78 2.3 8.0
The Netherlands 72 1.0 6.9
Portugal 10 0.9 14.2
Slovenia 10 3.0 9.5
Bulgaria 9 1.6 11.6
Serbia 8 -1.8 22.2
Spain 6 1.4 24.7
Greece 3 0.7 26.3

  1. Bernanke's hypothesized explanation is that "greater social and political polarization itself has a role to play in explaining reported levels of dissatisfaction. To an increasing extent, Americans are self-selecting into non-overlapping communities (real and virtual) of differing demographics and ideologies, served by a fragmented and partisan media. We see, for example, a sharply widening partisan gap in presidential approval ratings". Sahm suspects that "meager income growth for many could be playing a role" could be causing citizens' unhappiness with economic policy. ↩︎

  2. If you use a model with a quadratic term, then the following holds: $$ \frac{\partial Economy}{\partial Unemp} <0 \iff -10.27 + 2 \times 0.255 Unemp < 0 $$ which holds only as long as the reported unemployment rate is below 20.12%. After that threshold is passed, the model would imply that higher unemployment increases the likelihood of a positive assessment of the economy, a non-sensensical result. ↩︎

  3. Citizens of Austria, Czech Republic, and the UK assess their economy less favorably than the tree predicts, but the average error is close to zero. ↩︎

  4. If you multiply the shares of respondents by 100 so that your outcome variable is bounded by 100 - instead of 1 - then the residual mean deviance will be 189.5 = 5306 / 28. ↩︎

Michel Rocard

Former prime minister of France, Michel Rocard, passed away on July 2, 2016. New York Times obituary: “Our social system kills the goose that lays the...

How to have a respectful conversation

Recently I have seen three fine articles on how language can be barrier to honest and constructive communication. I am posting a brief a summary and...