A Stock Screen That Worked Last Year Can Quietly Stop Working

Most screeners list your saved screens with a date next to each one. Look at it. If the date is well over a year old and the filters haven’t been touched since, the screen is still answering a question about a market that no longer exists.

That’s the position. A filter set gets built, tested and saved at a particular moment, usually after a stretch when it would have surfaced some big winners. Those winners are part of why it looks good. The filters were chosen, consciously or not, to catch the kind of stock that had just done well, and there’s no guarantee the next stretch rewards the same kind.

So review saved screens on a schedule. People push back on this, usually in one of three ways.

“It made money”

It did. In one market, over one stretch, on a handful of trades.

Count them. A screen that produced eight positions last year, six of which worked, has a small sample behind it. Eight results can’t separate a good filter set from a lucky one, especially when all eight came from the same kind of market, where one broad force, falling rates or a rally in one sector, might have lifted almost anything the filters caught along with the stocks you’d have picked anyway.

Compare it with doing nothing clever, too. If a plain index fund rose about as much over the same stretch, the screen may have added little beyond the market’s own move.

The honest reading is narrower. The screen worked in that regime. Whether it works in the next depends on whether the reasons it worked still hold, and that’s something you check, which is the whole point of a review.

“Value always comes back”

Maybe. Not on your timeline.

Factors like value, momentum, quality and size have each gone through long stretches of doing poorly and long stretches of doing well, and they don’t all behave the same way in every environment. A value screen can go years surfacing cheap stocks that stay cheap. A momentum screen can work beautifully in a trending market and then get chopped apart when trends reverse. Believing a factor will eventually recover is a view about decades. Your screen needs to be useful this year.

There’s a second problem inside the objection. “Value” in a saved screen means a specific set of cutoffs: a P/E below some number, price to book below another, perhaps a dividend yield floor. Even if value as a broad idea comes back, those particular cutoffs may be catching a different mix of companies than they did when you set them, because the whole market’s multiples have moved.

“Changing it is overfitting too”

Yes. This is the strongest objection, and it’s right.

If you review a screen, see which stocks worked last quarter, and tweak the filters until those stocks appear, you have just done the same thing that made the original screen suspect. Tuning to recent results is overfitting whether it happens once or every quarter.

The answer is to change the screen only for a reason, and to write the reason down before you look at how the change would have performed. Good reasons look like this:

• A filter is excluding a type of company you meant to include.
• A cutoff no longer means what it did, because market-wide levels have shifted.
• The screen’s output has become too long or too short to research properly.
• A data field changed definition on your screener.

“The screen would have caught three more winners” is not on that list. It’s the one reason that guarantees the next version fits the past a little more tightly.

How to review a saved screen

Keep a log. Every time you run the screen, record the date, the stocks it surfaced, which ones you researched, which you bought, and what happened to each over the period you’d normally hold. A spreadsheet does the job.

After a few runs, the log shows things memory won’t. Whether the screen keeps producing names you end up rejecting. Whether the ones you buy do better than the ones you pass on. Whether its output has quietly shrunk to two or three stocks, which usually means a cutoff has drifted out of line with the market.

Then check what the filters exclude. Run the screen with one filter removed at a time and look at what comes back. A filter that removes most of the list deserves a hard look, since it may be doing the real work. Filters that exclude nothing are dead weight. A firm liquidity minimum is usually worth keeping, because it screens for tradability; most other cutoffs are worth questioning. A walkthrough of how to use a stock screener covers testing filters one at a time, and a screener with a saved-run history makes the log much easier to keep, which is one feature to look for in the screener buyer’s guide.

Where the rule relaxes

Simple quality screens change slowly. A screen for companies with consistent profitability, low debt and steady returns on capital is looking for traits that don’t swing much from year to year, and its cutoffs don’t depend heavily on where the market is trading. It still deserves an occasional look. The schedule can be longer.

The screens that need the closest watch are the clever ones: many filters, tight cutoffs, and a backtest that looks too good. Those are the most likely to have been tuned to one market, and the least likely to tell you when that market has gone.