What is evidence-based investing?

You might have heard the phrase “evidence-based investing” before – in truth, “evidence-based” is something of a buzz phrase which has been applied to many different activities that require some kind of decision making, such as medicine, safety training, and business management.

So in an investing context, what exactly does it mean? After all, most investment professionals would claim to base their philosophy on some form of evidence. But that doesn’t mean they’re worth paying any attention to.

In fact it’s remarkably easy to come up with an idea and then to claim it’s backed by rigorous evidence when, in fact, it’s nothing of the sort.

In 2015, for example, a journalist named John Bohannon and two German TV producers set out to demonstrate how easy it is to turn bad science into the big headlines behind diet fads. evidence based investingTheir challenge was to persuade as many media outlets as possible to publicize the results of “research” which “proved” that eating chocolate every day helps you lose weight.

So they spent a few thousand euros recruiting research subjects and a doctor to run the study. They intentionally used a tiny data set of 15 participants and just three weeks of data. Over a beer-fueled weekend with a statistician friend, they tortured the data sufficiently heavily to extract a technically accurate if essentially meaningless conclusion that chocolate consumption had indeed contributed to weight loss.

They submitted their paper to 20 different journals, and a few fake ones too, and within 24 hours the story was accepted for publication by dozens of publications around the globe. The study even made television news in the US and Australia.

Financial news desks are bombarded with “research” all the time, and very often it’s about as scientific as John Bohannon’s study on chocolate and weight loss. Almost invariably it has been produced or commissioned by a company with a commercial conflict of interest.

But the problem with bogus evidence in the investment industry runs far deeper. That’s because the research that investment advisers use to base their decisions on which stocks to buy and sell is, to a large extent, itself conflicted. Companies struggling for attention have been known to pay for the production of what looks like research by their broker. Those brokers often take payment for their work in warrants, or the right to buy the stock they have promoted at a future date at a substantial discount.

For investment advisers, like journalists, it can be very hard to distinguish between research that is properly independent and research that isn’t. It’s often only at the end of a report that the truth is revealed, with a disclaimer along the lines of, “This material is not investment research”.

So, then, what sort of evidence can investors rely on? In short, it’s academic evidence — but no, not all academic evidence. Generally speaking, reliable evidence has four main characteristics.

1. It’s independent

New studies are being published all the time which appear to support a particular investment strategy or course of action. But all too often these reports are produced, or else commissioned and paid for, by companies with a commercial interest in publicizing the outcome. Most academics, by contrast, are independent. They don’t have an agenda or a point to prove; instead they leave it to financial practitioners to act on their findings or not.

2. It’s based on robust data analysis

We all know the old adage about lies, damned lies and statistics. It’s true, abuse of data can be very misleading. Often findings are based on too short a time period or a sample that’s too small. Sometimes the investment industry ignores survivorship bias; in other words, it overlooks investments which performed so poorly that they no longer exist. Other times it compares returns to the wrong benchmark, or it quotes performance figures before the full impact of fees and charges. Sometimes it simply gets its math wrong.

3. It’s been peer-reviewed

To test whether their findings are reliable, academics publish their research in credible academic journals. This gives other academics the chance to agree or disagree on whether the results are sound. Again, caution is required — there are journals that are less credible than others — but evidence that has been properly peer-reviewed should carry far more weight with investors than evidence that hasn’t.

4. The results have been reproduced

The fourth and final characteristic of findings you can depend on is that they’ve been tested across multiple environments and timeframes. There is some disagreement on the extent to which academic finance is properly scientific. Asset prices, for example, never move in exactly the same way as they have done in the past. However there does need to be a strong element of repeatability to demonstrate that the findings of a particular study weren’t just down to random luck or else reached through “data mining”.

The lesson then is to be very cautious about anyone claiming that a particular investment strategy is evidence-based. Even if it fails just one of these four important tests, it’s probably worth ignoring.

Rethinking the value of wealth services

You'd Be Amazed at How Superstitious Investors Are

Interview: Risk Capacity

WealthFactor Review & Update: January 2021

What is evidence-based investing?

WealthFactor Review & Update: December 2020

"Don't just do something. Stand there!"

Interview: Does the financial media help or hurt?

Digital Wealth Management - Launch Announcement

Letting fear lead investment decisions can be costly

WealthFactor Review & Update: November 2020

Not all Advisors are Equal

Successful Investing Is Dull

Complexity Adds to Cost and Risk

WealthFactor October 2020 Review

Reducing market volatility through diversification

Taking a RiskSmart efficiency centric approach to investing.

It’s important to keep the link between wealth and happiness in mind.

September 2020 Review Replay

Preparing for your work-optional stage isn't just financial

August 2020 Review Replay

Webinar Replay: Introduction to Direct Indexing

Using Options for Income

Behavioral Investing Mistakes: Chasing Trends

July 2020 Review Replay

Emotional Investing Mistakes: Attention Span

Retail Investor Rises Again

Emotional Investing Mistakes: Regret

Perspective on Recessions

June 2020 Review Replay

Be Ready for Declines

There is always a reason not to invest

RiskSmart Webinar Replay

Don't work with professionals in the mirage building business.

Your Financial Advisor Has Too Many Clients

May 2020 Review Replay

Be Skeptical of the Investment Media

Beware of Overconfidence Bias

11 Ways Wall Street Gets Paid

Webinar Replay: An Introduction to Direct Indexing

What’s your number?

Refocus: What’s Wall Street Selling Today?

Do You Have a Plan for Getting Punched in the Face?

Financial Planning Matters. But...

WealthFactor April 2020 Review

Refocus 5-8-2020

2020 Berkshire Shareholder Meeting Key Takeaways

Why High Fees Hurt So Much

Refocus 5-1-2020

Introduction to Direct Indexing

Webinar: A Guide to RiskSmart Investing

Refocus 4-24-2020

Overview: PortfolioSmart

Overview: TaxSmart

Overview: FeeSmart

Overview: RiskSmart Investing

Webinar: The Critical Importance of an Investment Plan

Maximizing Opportunity with an Investment Plan - Part 3

The Critical Importance of having an Investment Plan Part #2:

Its time to turn off CNBC and create a plan.

1st Quarter 2020 Review Webinar Review

Why are you paying more and increasing your risk?

High Fees Are Hurting Your Investment Portfolio More Than the Coronavirus.

Pandemic Investing in 3 Parts: How to Navigate the Current Situation.

Four Behaviors Successful Investors Avoid

Are One of the Investors Who Underperform Passive Investing

Do You Invest With Your Emotions?

How should you invest?

Are You Paying Too Much For Investment Advice?

Have Questions? Do You Like Coffee?

Should You Use Funds or Stocks In Your Portfolio?

Commentary: Passive Investing Trends

Review & React: Goldman Sachs Fee Schedule!

Coffee Talk - Open Investment Q & A

Review & React: A Morgan Stanley Investment Proposal

RiskSmart Investing: Portfolio Efficiency

2019 Market and Investor Insights

RiskSmart Investing with WealthFactor

RiskSmart Investing with WealthFactor

Thank you to the Lake Oswego Review

New Addition to the WealthFactor Team

Quarterly Investor Insights

Sample - How To Post

Welcome to WealthFactor

Fed Intervenes in Banking Rate

Now You Can Blog from Everywhere!

Does your investment professional have history?

1st Quarter 2019 Insights

Keys To Investing Success

Tax-Smart Investors Avoid Mutual Funds

2018 Year-End Market Insights

Avoid Premium Prices for Commodity Services

2018 Mid-Year Market Insights

Ask your Adviser for a Discount

Index Investing Enhanced

Avoid Annuities

Avoid Excessive Risks With Bonds

FAANG Stocks are Adding Risk

Too Much Capital Gains Tax in 2017?

Custodians Overcharge

Trade, Taxes and Regulation

4 Reasons to Never Invest in a Mutual Fund

Risk-Smart by WealthFactor