Mean-Variance Optimization: Modern Portfolio Theory in Practice

Harry Markowitz's mean-variance optimization (MVO), introduced in 1952, remains the foundation of quantitative portfolio construction. Despite well-known limitations -- sensitivity to input estimates, concentration in estimation-error-favored assets, and the assumption that variance fully captures risk -- MVO provides the conceptual and mathematical framework upon which virtually all modern portfolio optimization methods are built. Understanding MVO thoroughly is prerequisite to understanding the methods that improve upon it.

The Mean-Variance Framework

Mathematical Formulation

An investor holds N assets with expected return vector mu (N x 1), and covariance matrix Sigma (N x N). The portfolio weight vector is w (N x 1).

The MVO problem minimizes portfolio variance for a given target return mu_target:

This is a quadratic programming problem with a closed-form solution when only equality constraints are present.

The Efficient Frontier

The set of all optimal portfolios (those that minimize variance for each level of expected return) forms the efficient frontier -- a curve in (sigma, mu) space. No portfolio below the frontier is optimal, because you can find another portfolio with the same return and lower risk, or the same risk and higher return.

The efficient frontier is a parabola in (variance, return) space and a hyperbola in (standard deviation, return) space. Two portfolios on the efficient frontier, combined in any proportion, produce another portfolio on the efficient frontier. This two-fund separation theorem means that every efficient portfolio can be expressed as a linear combination of any two distinct efficient portfolios.

The Tangency Portfolio

When a risk-free asset is available (with return R_f), the optimal risky portfolio is the tangency portfolio -- the portfolio on the efficient frontier with the highest Sharpe ratio:

The capital market line (CML) connects the risk-free asset to the tangency portfolio, and every optimal complete portfolio (combining risk-free and risky assets) lies on this line.

Practical Implementation

Input Estimation

MVO requires two inputs: expected returns (mu) and the covariance matrix (Sigma). The quality of these inputs determines the quality of the output. This is MVO's Achilles' heel.

Alternative approaches to expected return estimation:

• Equilibrium returns (CAPM/Black-Litterman): Use market capitalization weights to reverse-engineer implied expected returns. More stable than historical estimates.
• Factor model returns: Express expected returns as a function of factor exposures (value, momentum, quality) and factor risk premia.
• Shrinkage: Blend historical returns toward a common mean (grand mean, sector mean, or CAPM expected return).

Covariance estimation remedies:

• Ledoit-Wolf shrinkage: Shrink toward a structured target (constant correlation, single factor). Optimal shrinkage intensity is determined analytically.
• Factor models: Reduce dimensionality by decomposing returns into k factors (k << N). The covariance matrix is approximated as B F B^T + D, where B is the factor loading matrix, F is the factor covariance, and D is the diagonal idiosyncratic variance.
• Random Matrix Theory (RMT): Identify and remove eigenvalues of the sample covariance matrix that are attributable to noise rather than signal. Eigenvalues below the Marcenko-Pastur bound are replaced with the average noise eigenvalue.

Constraint Handling

Unconstrained MVO produces portfolios that are concentrated, leveraged, and impractical. Constraints improve portfolio quality:

Regularization

Adding a penalty term to the objective function reduces sensitivity to estimation error:

The L2 regularization term (ridge regression applied to portfolio optimization) shrinks weights toward zero, reducing concentration and improving out-of-sample performance. The regularization parameter lambda controls the trade-off between optimality and stability.

L1 regularization (LASSO) encourages sparsity, producing portfolios with fewer holdings. This is useful when transaction costs are high or operational simplicity is valued.

Resampled Efficiency

Richard Michaud's resampled efficient frontier addresses estimation error through Monte Carlo simulation:

1. Estimate mu and Sigma from historical data
2. Generate K simulated parameter sets by sampling from the distribution of the estimates
3. Solve the MVO problem for each simulated parameter set, generating K efficient frontiers
4. Average the portfolio weights across simulations for each risk level

Robust Optimization

Robust MVO explicitly accounts for uncertainty in the inputs by optimizing for the worst case within a specified uncertainty set:

The uncertainty set is typically an ellipsoid centered on the point estimates. The resulting portfolio is conservative -- it performs well even when the true parameters differ from the estimates by up to the uncertainty set boundary.

In practice, robust optimization produces portfolios similar to those from constrained MVO with tight position limits, suggesting that simple constraints capture much of the benefit of formal robustness.

MVO Extensions

Mean-Semivariance Optimization

Replace variance with semivariance (variance of returns below a threshold, typically the mean or zero):

This captures the intuition that investors care about downside risk, not upside volatility. The optimization is no longer a standard QP and requires specialized solvers.

Mean-CVaR Optimization

Replace variance with Conditional Value at Risk (see the Expected Shortfall article). This produces portfolios with lower tail risk, at the cost of solving a linear program rather than a quadratic program.

Multi-Period Optimization

Single-period MVO ignores the dynamic nature of investing. Multi-period optimization considers:

• Transaction costs from rebalancing
• Time-varying investment opportunities
• Intermediate consumption or liability payments
• Changing risk aversion over the investment horizon

Key Takeaways

• Mean-variance optimization finds the portfolio with minimum variance for a given expected return, producing the efficient frontier of optimal risk-return trade-offs
• The primary practical challenge is input estimation: expected returns are noisy, and the covariance matrix is poorly estimated when assets outnumber observations
• Shrinkage estimators, factor models, and regularization substantially improve out-of-sample portfolio performance by reducing sensitivity to estimation error
• Constraints (long-only, position limits, turnover limits) are not just practical requirements but serve as implicit regularization, improving portfolio stability
• Extensions including robust optimization, resampled efficiency, mean-semivariance, and mean-CVaR address specific limitations of classical MVO while preserving its core framework

Frequently Asked Questions

Why do MVO portfolios perform poorly out of sample?

MVO maximizes in-sample fit to the estimated parameters, which includes fitting to estimation noise. Assets with overestimated returns and underestimated risk receive the largest allocations -- precisely the assets most likely to disappoint. This is the "error maximization" property of MVO, first identified by Michaud (1989). The solution is not to abandon MVO but to improve input estimation and add constraints that prevent the optimizer from exploiting estimation errors.

What Sharpe ratio should I use as a target?

The tangency portfolio's Sharpe ratio depends entirely on the input estimates. Historically, diversified equity portfolios achieve Sharpe ratios of 0.3-0.5. Multi-asset portfolios with risk parity characteristics can achieve 0.5-0.8. Individual stocks rarely sustain Sharpe ratios above 0.2. If your optimizer produces a Sharpe ratio above 1.0 for a long-only equity portfolio, the expected return estimates are likely too aggressive.

How often should I re-optimize the portfolio?

Re-optimization frequency depends on turnover costs and how quickly the inputs change. Covariance estimates are relatively stable month-to-month, so monthly or quarterly re-optimization is typical. Expected return signals (momentum, value, etc.) may update daily, but the cost of daily turnover typically exceeds the benefit. A practical approach: re-optimize monthly, implement trades only when the weight change exceeds a threshold (e.g., 1% per position).

Can MVO handle alternative assets (private equity, real estate, hedge funds)?

MVO can technically handle any asset class, but the inputs are problematic for illiquid assets. Private equity returns are smoothed (reported infrequently), understating true volatility by 30-50%. Correlations with public markets are also understated. Adjustments include unsmoothing private asset returns (Geltner method) and using longer estimation windows. Even with adjustments, MVO tends to over-allocate to alternatives because their reported risk-return profiles appear artificially attractive.

What is the difference between MVO and risk parity?

MVO allocates capital to maximize expected return per unit of risk, which requires expected return estimates. Risk parity allocates risk equally across assets, requiring only the covariance matrix. When all assets have the same Sharpe ratio, the two approaches produce identical portfolios. In practice, risk parity avoids the estimation error problem associated with expected returns, producing more stable and diversified portfolios at the cost of not expressing views on relative asset attractiveness.