The Idiosyncratic Volatility Puzzle: Why Risky Stocks Earn Less

The Debate: Should Firm-Specific Risk Be Rewarded?

For decades, one of the central promises of the Capital Asset Pricing Model has been clean and intuitive: investors who bear more systematic risk should earn higher returns, and firm-specific (idiosyncratic) risk, because it can be diversified away, should carry no risk premium at all. Yet a competing view, rooted in Merton (1987), argues that incomplete information and underdiversified portfolios mean idiosyncratic risk does matter and should command positive compensation. These two positions frame one of the most contentious debates in empirical asset pricing. In 2006, Ang, Hodrick, Xing, and Zhang brought data to the argument and produced a result that neither side expected: stocks with the highest idiosyncratic volatility earned not higher returns, but dramatically lower ones.

Their paper, "The Cross-Section of Volatility and Expected Returns," published in The Journal of Finance, found that a portfolio of the most idiosyncratically volatile US stocks underperformed the least volatile quintile by roughly 1.06% per month. This was not a marginal statistical artifact. It was an economically massive spread that persisted after controlling for the Fama-French factors, momentum, liquidity, and a battery of other known return predictors. The finding posed a direct challenge to both the CAPM (which predicts no relationship) and to Merton's incomplete-information model (which predicts a positive relationship).

How the Puzzle Was Measured

Ang, Hodrick, Xing, and Zhang (2006) estimated idiosyncratic volatility using the residuals from a Fama-French three-factor regression fitted to daily stock returns. For each stock in each month, they ran the three-factor model over the prior month's daily data and computed the standard deviation of the unexplained returns. This measure captures the portion of a stock's day-to-day fluctuations that cannot be attributed to market, size, or value exposures.

Stocks were then sorted into quintile portfolios based on the prior month's idiosyncratic volatility, and equal-weighted returns were tracked for the following month. The results formed a monotonically declining pattern.

The spread between the lowest and highest IVOL quintiles was striking not only in magnitude but in consistency. It survived adjustments for the Fama-French three-factor model, the Carhart four-factor model, and additional controls for short-term reversal, liquidity, and volume. The Fama-French alpha of the high-IVOL quintile was deeply negative.

The authors confirmed that these results were not driven by micro-cap stocks, penny stocks, or extreme outliers. Excluding the smallest size decile, applying value-weighting, and trimming the most extreme observations all preserved the core finding: higher idiosyncratic volatility was associated with lower subsequent returns.

International Confirmation

A natural objection to any US-only anomaly is that it might reflect data-mining or a peculiarity of American market structure. Ang, Hodrick, Xing, and Zhang addressed this directly in a 2009 follow-up paper that extended the analysis to 23 developed equity markets. In every region they examined, from Japan and Australia to the United Kingdom and continental Europe, the relationship between idiosyncratic volatility and subsequent returns was negative. The magnitude varied, but the direction was consistent. A global portfolio that was long low-IVOL stocks and short high-IVOL stocks generated reliable positive returns across geographies.

This international replication substantially raised the burden of proof for any explanation that relied on US-specific institutional features, data construction artifacts, or sample-period coincidences.

Competing Explanations: A Field Divided

The two decades since the original paper have produced a rich and sometimes contradictory literature attempting to explain the puzzle. The explanations fall into three broad categories.

The Measurement Debate

Fu (2009) raised an influential challenge by arguing that Ang et al. used backward-looking (realized) idiosyncratic volatility as a proxy for forward-looking (expected) idiosyncratic volatility. When Fu estimated expected IVOL using an EGARCH model that accounts for volatility clustering, the negative relationship reversed: higher expected IVOL was associated with higher returns, consistent with Merton's theory. Fu's interpretation was that the Ang et al. result reflected a short-term return reversal effect, since stocks that recently experienced spikes in idiosyncratic volatility (and hence high realized IVOL) tend to revert in the following month.

Bali and Cakici (2008) offered a different methodological critique, demonstrating that the IVOL puzzle was sensitive to portfolio construction choices. Under value-weighting, the negative IVOL-return relationship weakened or disappeared for certain subsamples, suggesting that the effect was concentrated among small, illiquid stocks where equal-weighting exaggerated its economic importance.

These methodological objections are serious but not fully dispositive. Hou and Loh (2016) conducted a comprehensive assessment of over a dozen proposed explanations for the IVOL puzzle and found that no single explanation could account for more than about 40% of the anomalous spread. The measurement critique explains some, but not all, of the phenomenon.

The Behavioral and Structural Account

A second class of explanations attributes the puzzle to investor behavior and market structure. The most influential of these comes from Stambaugh, Yu, and Yuan (2015), who proposed "arbitrage asymmetry" as the key mechanism. Their argument proceeds in two steps. First, high-IVOL stocks are harder and costlier to sell short because they tend to be smaller, less liquid, and more expensive to borrow. Second, when short-selling is constrained, stocks that are overpriced stay overpriced longer than stocks that are underpriced, because buying underpriced stocks faces no analogous barrier.

The implication is that the pool of high-IVOL stocks contains a disproportionate share of overpriced names. These overpriced stocks drag down the average return of the high-IVOL quintile. The negative IVOL-return relationship is not a compensation for risk at all; it is the footprint of uncorrected overpricing among the most volatile segment of the market.

Stambaugh et al. tested this by decomposing the IVOL effect into overpricing and underpricing components. Among stocks classified as overpriced (using 11 anomaly signals), high IVOL strongly predicted low returns. Among underpriced stocks, high IVOL predicted higher returns, exactly as Merton's theory would suggest. The negative average relationship arose because the overpricing effect dominated.

This explanation connects directly to the broader low-volatility anomaly, which documents that calm stocks outperform volatile ones on a risk-adjusted basis. The Stambaugh et al. framework suggests that both phenomena share a common root: the asymmetric ability of markets to correct overpricing versus underpricing.

Lottery Demand and Speculative Preference

A third strand of research highlights the role of investor preferences for lottery-like payoffs. Stocks with high idiosyncratic volatility tend to have positively skewed return distributions: they occasionally produce dramatic gains. If a subset of investors is willing to overpay for the chance of extreme upside, as documented in the behavioral finance literature on prospect theory and cumulative prospect theory, then lottery-like stocks will be persistently bid above fundamental value. The resulting overpricing depresses their expected returns.

Bali, Cakici, and Whitelaw (2011) formalized this channel by constructing a variable, MAX, defined as the maximum daily return in the prior month. They showed that MAX absorbed much of the IVOL puzzle: stocks with high IVOL earned low returns primarily because they also had high MAX, and it was the lottery demand for extreme-gain stocks that drove the overpricing.

What Hou and Loh Found When They Tested Everything

Hou and Loh (2016) attempted the most systematic resolution of the puzzle to date. They evaluated explanations based on: (1) expected vs. realized IVOL measurement, (2) return reversals, (3) short-selling constraints, (4) MAX/lottery demand, (5) market microstructure noise, (6) earnings surprises, (7) leverage effects, and several other channels.

Their conclusion was sobering: no single explanation accounted for more than 30-40% of the IVOL effect. The largest individual contributors were the lottery demand (MAX) channel and the short-selling constraints channel. When combined, a handful of explanations could collectively account for roughly 60-80% of the puzzle, but a residual remained unexplained. The idiosyncratic volatility puzzle, in their assessment, was partially solved but not fully resolved.

Implications for Factor-Based Investing

The IVOL puzzle carries concrete implications for anyone constructing or evaluating factor portfolios. First, it provides an additional rationale for low-volatility strategies beyond the traditional betting-against-beta framework. Screening out high-IVOL stocks removes a pocket of the market that has historically delivered deeply negative alpha, regardless of the economic mechanism responsible.

Second, the puzzle interacts with other factors in important ways. High-IVOL stocks tend to be small, unprofitable, and have high investment rates, placing them at the intersection of several negative-return characteristics identified in the Fama-French five-factor model. Investors who tilt toward quality, profitability, or conservative investment are implicitly avoiding high-IVOL stocks.

Third, the arbitrage asymmetry explanation has implications for how investors should interpret factor backtest results. If high-IVOL short legs are responsible for a large share of factor returns (as Stambaugh et al. demonstrate), then long-only factor investors are systematically missing the portion of the premium that comes from avoiding overpriced, volatile stocks. The IVOL puzzle thus reinforces a broader lesson: the gap between theoretical factor premiums and implementable returns is real and persistent.

Where the Debate Stands

Twenty years after Ang, Hodrick, Xing, and Zhang published their original finding, the community has converged on a partial consensus. The IVOL puzzle is real and robust. It appears in US data, international data, and across multiple sample periods. It is not fully explained by any single mechanism, but the combination of arbitrage asymmetry (overpricing that cannot be corrected due to short-selling constraints), lottery demand (speculative investors overpaying for skewed payoffs), and methodological nuance (the distinction between realized and expected IVOL) accounts for the majority of the effect.

What remains genuinely unresolved is whether any residual portion of the puzzle reflects a true negative risk premium, a possibility that would be deeply uncomfortable for standard asset pricing theory. If investors are somehow penalized for holding idiosyncratic risk, the entire framework that separates systematic from diversifiable risk would need revision. Most researchers consider this unlikely but have not fully ruled it out.

For practitioners, the operational conclusion is clearer than the theoretical one: high-idiosyncratic-volatility stocks have been poor investments on average, and the evidence is strong enough to merit consideration in portfolio construction and risk management, even as the academic debate over root causes continues.

References