folio

Fixed before the first fit

Method

The rules were written down before any model was fitted, so a result cannot quietly change what counts as success.

What is forecast

For each of eleven US sector funds and each month end: the fund's total return over the following month, minus that of the S&P 500 fund.

What the model may see

A forecast made at a month end uses only figures that were public on or before that day. A closing price counts on its own day; a release or a filing counts from the day after it came out. Where a figure was later revised, the model sees the first-published value, and the revision only once it was made. A test cuts the data off at a month end and requires every earlier row to be unchanged.

The model

Gradient-boosted decision trees, one model for all sectors, refitted every month on all earlier months. Its settings (300 trees, learning rate 0.02, depth 3, at least 20 rows per leaf, 70% of rows and 50% of inputs per tree) were fixed in advance and are not tuned on the months it is scored on. Each forecast is the average of 10 fits.

The score

Each month, the rank correlation across sectors between the forecast and what happened: 1 is a perfect ranking, 0 is chance, −1 is exactly backwards. The headline is its average over all scored months, with a bootstrap interval. Also reported: the return of the three highest-ranked sectors minus the three lowest.

The model counts as working only if the interval excludes zero and the score is above that of simple momentum (ranking sectors by their return over the past year). Anything else is reported as not shown to work.

Guards against fooling ourselves

  • Held-back months. The 36 months from October 2023 are not scored or looked at while the model is being built. They are read once, after it is frozen.
  • A count of tries. Every fit is logged. The interval is widened for the number of configurations tried, so trying many things costs something.
  • Source groups in the open. The score with each group of inputs removed is published, so a group that adds nothing is visible.

Limits

Sector funds exist only since 1998, which is about three market cycles. Many inputs start later and are simply missing before then. Published research finds effects of this kind to be small. Costs and tax are not part of forecast scoring. This is statistical model output, not investment advice.