Take the 47 Nifty 50 stocks with clean price histories from 10 Sep 2025 to 15 Sep 2026. Every group of three is a possible basket, 16,215 of them. I split the year in two and ran a five percent cointegration test on every basket in the first half. About one basket in nineteen passed, almost exactly what luck alone would give. Then I ran the same test on the second half. The baskets that had passed did not pass more often than the rest, they passed slightly less often. This lesson moves from pairs to baskets: what principal components separate, what the Johansen test asks, and what a screen of thousands tells you about itself. Dated NSE data, nothing live, nothing a view.
3By the end you can
- Explain how principal components separate shared moves from a residual, and how the noise edge limits the count.
- Read a Johansen trace statistic against cutoffs that match the sample length.
- Say what a screen of thousands of baskets finds, and what to count before trusting a pass.
The idea, plainly
Why more than two names
The cointegration toolkit tested one pair. A pair depends on a single partner, and if that partner has news, the pair breaks. A basket spreads the dependence. The idea of statistical arbitrage on baskets is to take the move that all stocks share out of each stock, and then look at what is left. The leftover is a stock's own part: its news, its flows, its noise. If some of the leftover tends to drift back, a rule can try to harvest it. Whether it drifts back is the question, and this lesson tests it on dated data.
Principal components: the shared moves
Take daily returns of the stocks, standardise each, and form the correlation matrix. Its eigenvectors are the shared moves, called factors, and its eigenvalues say how much of the total variance each one carries. The first is nearly always the market: every stock loads on it with the same sign. The next ones are blocks of stocks that move together beyond the market, such as a sector.
The maths: factors and the noise edge
Let Z hold standardised returns, T rows by N columns.
R = Z'Z / T
R v_k = lambda_k v_k
F_k = Z v_k
edge = (1 + sqrt(N/T))^2
The eigenvalues of R sum to N, so lambda_k / N is the share of variance of factor k. The edge is the largest eigenvalue pure noise would give as T grows. It is an asymptotic result, so with a few hundred sessions it is a guide, not a line.
Strip out the shared moves, then ask whether the leftover reverts
Read one more paragraph, 50 words
The hard part is how many to keep. A random matrix of the same size also produces eigenvalues, and the largest of them is not zero. For N stocks and T sessions the noise edge is known in closed form. Only eigenvalues above it are likely to be real structure.
The residual and its speed
With the factors in hand, regress each stock on them. What the regression leaves unexplained is the stock's residual return. Add the residuals day by day and you get a path. If that path is pulled back toward a mean, it has a speed, and the speed gives a half life: how long half of a gap takes to close. The distance from the mean in units of the path's usual size is a score. A cheap name has a low score, a rich one a high score, and a rule goes long the first and short the second.
The maths: residual path, half life, score
r_i = a + sum b_k F_k + e_i
X_t = X(t-1) + e_t
X_t = c + b X(t-1) + z_t
half life = -ln 2 / ln b
s = (X-m) / sd_eq
The second line is the cumulative residual. The third is an AR(1) fit of that path, where b below one means it reverts. The equilibrium spread is sd_eq, the standard deviation of z divided by sqrt(1 - b^2). Everything depends on the window the fit used.
Johansen: a test made for baskets
The Engle Granger test of the toolkit lesson regresses one price on the others and tests the leftover. For three names it gives a different answer depending on which one you put on the left. The Johansen test treats all the prices alike. It asks how many independent combinations of them are stationary. That number is the rank r. If r is zero the prices have no home. If it is one there is one combination that keeps returning, with weights the test also gives you.
The maths: rank and the trace statistic
dY_t = Pi Y(t-1) + c + e_t
Pi = alpha beta'
trace = -T sum ln(1-lam_i)
sum over i above r
Pi is the matrix that pulls the prices back. Its rank is r. The lambda_i come from a matrix built from the regression residuals, and the trace statistic for r equal to zero uses all of them. The statistic is compared with cutoffs. The published tables assume a long sample, so here I simulated the cutoffs for each sample length, drawing random walks and running the same test.
3 terms this section turns on
Worked example: a year of Nifty 50 stocks
How many factors are real
On the last two hundred and fifty daily returns to 15 Sep 2026, the first factor carries about three tenths of all the variance in the file. That is the market. The chart shows the first five eigenvalues against the noise edge.
Three factors rise above the noise edge; the rest cannot be told from noise
Read one more paragraph, 54 words
Three factors clear the edge and the fourth falls just short of it. The second factor is dominated by the IT names. The third pulls the metals, power and energy stocks against the consumer and finance ones. A basket rule that keeps eight factors is keeping five that this sample cannot tell from noise.
Do the residuals revert?
Now the test that matters. On each day from Feb 2026 I fit a factor model on the previous hundred sessions, found each stock's residual over the last one, five or ten days, then went long the five most negative and short the five most positive for one day. I did it with one, three, five and eight factors, and with a one, five and ten day look back. That is twelve settings, and I report all of them.
Five of twelve settings stayed positive after statutory costs, which is what a search looks like
Read one more paragraph, 131 words
Gross of costs the rule averaged between nothing and about sixteen basis points a day, depending on the setting, with a daily standard deviation of roughly a hundred. At that noise a mean of ten basis points over a hundred and forty days is about one standard error, and the best setting reached a t statistic of about two among twelve tries. If the twelve settings were unrelated noise, about one search in four would still turn up a t that large somewhere among them. The signal itself, the correlation of residual score with the next day's residual return, was small and positive at about three hundredths. That is a faint reversal. It is not a result, and the statutory costs alone take most of the shorter look backs below zero.
Johansen on a basket of three large banks
Take three large banks, with their log prices for the same year. The trace statistic of the whole sample is above the five percent cutoff for its length. Split the year and neither half is.
The basket passed on the whole year and on neither half
Read one more paragraph, 50 words
The weights of the combination moved too. The weight on the second bank was negative in the first half and large and positive in the second. A relationship that reverses sign has no hedge ratio to hold. The module on Kalman hedge ratios asks whether letting the weights drift helps.
Screening every basket
One basket proves little. All of them show what a screen does. I ran the test on every one of the sixteen thousand baskets in each half.
Of the baskets that passed once, fewer than chance passed again
Read one more paragraph, 91 words
The first half passed five and a quarter percent of baskets, close to the five percent the cutoff promises, so the cutoffs behave as they should on this sample. The second half passed fewer. Of the baskets that passed in the first half, under two in a hundred passed again, below the rate for baskets overall. Choosing the best baskets from the first half and trading them in the second would have picked nothing. That is the answer to a screen of thousands: it finds what the data happened to do.
What this test leaves out
Three of the fifty stocks were left out. Two had a one day price move of more than a quarter inside the window, which in an end of day file is usually a split, a bonus issue or a demerger and not a real return, and one did not have a full history. Prices are the cash underlying, not futures, so they carry no basis and no roll. Dividends sit inside the returns: an ex-dividend drop looks like a negative residual that needs no explanation. The rule always held five names a side, equal weight, and took the closing prices as fills. None of that is how a desk would trade it, and each omission flatters the result a little. A result that is weak with the flattering version is weaker with the realistic one.
Where this meets India
A basket rule holds many legs, and for a retail account each short leg is in practice a stock future, since a cash short without a borrowed share must be bought back the same day. Each future has its own lot, its own monthly expiry and delivery on the last day. Whole lots do not fit a hedge ratio, as the stock futures pair lesson showed for two legs, and ten legs round worse. A leg can enter the ban list. The statutory stack costs about the same for each leg round trip as for a pair, so a rule that turns half its names over every day pays it on every one of them. The F&O stock list also changes: SEBI revised the entry and exit tests in Aug 2024, so a basket drawn from today's list is not a basket of a year ago.
Appendix: how the cutoffs were made
The published trace cutoffs are for a very long sample. For a sample of a hundred and twenty five sessions the true cutoff is higher. I drew three thousand random walks of the exact length with a small drift, three series each, ran the same trace statistic on them, and took the ninety fifth percentile. As a check, the first half passed five and a quarter percent of baskets, and a long sample of five thousand sessions reproduced the published cutoffs to within about half a point. The numbers are a method of mine, not a standard.
The common mistake
The claim: This basket of stocks passed a cointegration test, so the spread has to come back.
A pass is a statement about one sample, at an error rate. Across many baskets, some pass by luck, and in this file those that passed in the first half did not pass again in the second. The weights of the basket moved with the window, and the factors that a rule strips out are in part noise below the edge. Before taking a basket seriously, count how many were screened, hold the weights fixed on days the search never saw, include statutory costs for every leg, and ask how many settings were tried.
Key takeaways
- Principal components strip out the shared market move, leaving each stock's residual.
- Only factors above the noise edge are likely to be real.
- The Johansen test asks how many stationary combinations a basket holds.
- Baskets that passed a test once did not pass again more often than chance.
- The weights of a basket can change sign between windows.
- Count the baskets screened and the settings tried before trusting any pass.
Two terms to hold on to from 18.13
Before you trade. Derivatives trading carries a high risk of loss. SEBI's study "Profitability of Individual Traders in the Equity Derivatives Segment (FY25 to FY26)" (August 2026) found that 87.7% of individual traders in equity F&O made net losses in FY26, with aggregate net losses of Rs 91,685 crore. Read the SEBI study