--- title: "PCA Stat Arb and Johansen Tests on Nifty 50 Stocks" description: "Principal components, residual reversion and the Johansen test on a year of Nifty 50 stocks, with every setting reported and what a basket screen finds." source_url: "https://www.rupeecase.com/learn/path-18/module-18-13-pca-stat-arb" --- [Path 18 13 of 14](/learn/path-18/)Baskets, PCA stat arb and Johansen tests[Next 18.14](/learn/path-18/module-18-14-kalman-filter-hedge-ratios) Take the 47 Nifty 50 stocks with clean price histories from 10 Sep 2025 to 15 Sep 2026. Every group of three is a possible basket, 16,215 of them. I split the year in two and ran a five percent cointegration test on every basket in the first half. About one basket in nineteen passed, almost exactly what luck alone would give. Then I ran the same test on the second half. The baskets that had passed did not pass more often than the rest, they passed slightly less often. This lesson moves from pairs to baskets: what principal components separate, what the Johansen test asks, and what a screen of thousands tells you about itself. Dated NSE data, nothing live, nothing a view. 3By the end you can 1. Explain how principal components separate shared moves from a residual, and how the noise edge limits the count. 2. Read a Johansen trace statistic against cutoffs that match the sample length. 3. Say what a screen of thousands of baskets finds, and what to count before trusting a pass. ## The idea, plainly ### Why more than two names The [cointegration toolkit](/learn/path-18/module-18-7-cointegration-toolkit) tested one pair. A pair depends on a single partner, and if that partner has news, the pair breaks. A basket spreads the dependence. The idea of statistical arbitrage on baskets is to take the move that all stocks share out of each stock, and then look at what is left. The leftover is a stock's own part: its news, its flows, its noise. If some of the leftover tends to drift back, a rule can try to harvest it. Whether it drifts back is the question, and this lesson tests it on dated data. ### Principal components: the shared moves Take daily returns of the stocks, standardise each, and form the correlation matrix. Its eigenvectors are the shared moves, called factors, and its eigenvalues say how much of the total variance each one carries. The first is nearly always the market: every stock loads on it with the same sign. The next ones are blocks of stocks that move together beyond the market, such as a sector. The maths: factors and the noise edge Let Z hold standardised returns, T rows by N columns. R = Z'Z / T R v_k = lambda_k v_k F_k = Z v_k edge = (1 + sqrt(N/T))^2 The eigenvalues of R sum to N, so lambda_k / N is the share of variance of factor k. The edge is the largest eigenvalue pure noise would give as T grows. It is an asymptotic result, so with a few hundred sessions it is a guide, not a line. Strip out the shared moves, then ask whether the leftover reverts The usual order of a factor based basket rule. Everything up to the last box is estimated on a formation window, and the last box uses days the estimate never saw. An illustration of the method, with no prices. Source: RupeeCase illustration of the method. As of 17 Oct 2026. Read one more paragraph, 50 words The hard part is how many to keep. A random matrix of the same size also produces eigenvalues, and the largest of them is not zero. For N stocks and T sessions the noise edge is known in closed form. Only eigenvalues above it are likely to be real structure. ### The residual and its speed With the factors in hand, regress each stock on them. What the regression leaves unexplained is the stock's residual return. Add the residuals day by day and you get a path. If that path is pulled back toward a mean, it has a speed, and the speed gives a half life: how long half of a gap takes to close. The distance from the mean in units of the path's usual size is a score. A cheap name has a low score, a rich one a high score, and a rule goes long the first and short the second. The maths: residual path, half life, score r_i = a + sum b_k F_k + e_i X_t = X(t-1) + e_t X_t = c + b X(t-1) + z_t half life = -ln 2 / ln b s = (X-m) / sd_eq The second line is the cumulative residual. The third is an AR(1) fit of that path, where b below one means it reverts. The equilibrium spread is sd_eq, the standard deviation of z divided by sqrt(1 - b^2). Everything depends on the window the fit used. ### Johansen: a test made for baskets The Engle Granger test of the toolkit lesson regresses one price on the others and tests the leftover. For three names it gives a different answer depending on which one you put on the left. The Johansen test treats all the prices alike. It asks how many independent combinations of them are stationary. That number is the rank r. If r is zero the prices have no home. If it is one there is one combination that keeps returning, with weights the test also gives you. The maths: rank and the trace statistic dY_t = Pi Y(t-1) + c + e_t Pi = alpha beta' trace = -T sum ln(1-lam_i) sum over i above r Pi is the matrix that pulls the prices back. Its rank is r. The lambda_i come from a matrix built from the regression residuals, and the trace statistic for r equal to zero uses all of them. The statistic is compared with cutoffs. The published tables assume a long sample, so here I simulated the cutoffs for each sample length, drawing random walks and running the same test. 3 terms this section turns on From the module glossary; every term is in the Learn glossary too. Source: Module 18.13, Baskets, PCA stat arb and Johansen tests. ## Worked example: a year of Nifty 50 stocks ### How many factors are real On the last two hundred and fifty daily returns to 15 Sep 2026, the first factor carries about three tenths of all the variance in the file. That is the market. The chart shows the first five eigenvalues against the noise edge. Three factors rise above the noise edge; the rest cannot be told from noise Eigenvalues of the correlation matrix of daily returns. The grey bar is the noise edge for this many stocks and sessions. The first factor is the market, the second an IT block, the third a split between metals, power and energy names and consumer and finance names. Source: NSE end of day data, computed by RupeeCase. As of 15 Sep 2026. Read one more paragraph, 54 words Three factors clear the edge and the fourth falls just short of it. The second factor is dominated by the IT names. The third pulls the metals, power and energy stocks against the consumer and finance ones. A basket rule that keeps eight factors is keeping five that this sample cannot tell from noise. ### Do the residuals revert? Now the test that matters. On each day from Feb 2026 I fit a factor model on the previous hundred sessions, found each stock's residual over the last one, five or ten days, then went long the five most negative and short the five most positive for one day. I did it with one, three, five and eight factors, and with a one, five and ten day look back. That is twelve settings, and I report all of them. Five of twelve settings stayed positive after statutory costs, which is what a search looks like Underlying prices, one day holds, the five lowest and five highest residual scores each side, a little under a hundred and fifty test days. Statutory costs follow each setting's own turnover, at the futures leg stack of the pair lesson. Brokerage and price impact are not included. Source: NSE end of day data and NSE statutory charges, computed by RupeeCase. As of 15 Sep 2026. Read one more paragraph, 131 words Gross of costs the rule averaged between nothing and about sixteen basis points a day, depending on the setting, with a daily standard deviation of roughly a hundred. At that noise a mean of ten basis points over a hundred and forty days is about one standard error, and the best setting reached a t statistic of about two among twelve tries. If the twelve settings were unrelated noise, about one search in four would still turn up a t that large somewhere among them. The signal itself, the correlation of residual score with the next day's residual return, was small and positive at about three hundredths. That is a faint reversal. It is not a result, and the statutory costs alone take most of the shorter look backs below zero. ### Johansen on a basket of three large banks Take three large banks, with their log prices for the same year. The trace statistic of the whole sample is above the five percent cutoff for its length. Split the year and neither half is. The basket passed on the whole year and on neither half A grey bar is the simulated five percent cutoff for a sample of that length, from random walks run through the same test. A statistic must reach it to reject the idea that the prices have no home. The whole year barely reached it. Source: NSE end of day data, computed by RupeeCase; cutoffs simulated by RupeeCase. As of 15 Sep 2026. Read one more paragraph, 50 words The weights of the combination moved too. The weight on the second bank was negative in the first half and large and positive in the second. A relationship that reverses sign has no hedge ratio to hold. The module on [Kalman hedge ratios](/learn/path-18/module-18-14-kalman-filter-hedge-ratios) asks whether letting the weights drift helps. ### Screening every basket One basket proves little. All of them show what a screen does. I ran the test on every one of the sixteen thousand baskets in each half. Of the baskets that passed once, fewer than chance passed again Forty seven stocks, the first half of the sessions from 10 Sep 2025 and the second half to 15 Sep 2026. The grey bar is the count of baskets passing both halves if passing the first half told nothing about the second. Source: NSE end of day data, computed by RupeeCase. As of 15 Sep 2026. Read one more paragraph, 91 words The first half passed five and a quarter percent of baskets, close to the five percent the cutoff promises, so the cutoffs behave as they should on this sample. The second half passed fewer. Of the baskets that passed in the first half, under two in a hundred passed again, below the rate for baskets overall. Choosing the best baskets from the first half and trading them in the second would have picked nothing. That is the answer to a screen of thousands: it finds what the data happened to do. ### What this test leaves out Three of the fifty stocks were left out. Two had a one day price move of more than a quarter inside the window, which in an end of day file is usually a split, a bonus issue or a demerger and not a real return, and one did not have a full history. Prices are the cash underlying, not futures, so they carry no basis and no roll. Dividends sit inside the returns: an ex-dividend drop looks like a negative residual that needs no explanation. The rule always held five names a side, equal weight, and took the closing prices as fills. None of that is how a desk would trade it, and each omission flatters the result a little. A result that is weak with the flattering version is weaker with the realistic one. ### Where this meets India A basket rule holds many legs, and for a retail account each short leg is in practice a stock future, since a cash short without a borrowed share must be bought back the same day. Each future has its own lot, its own monthly expiry and delivery on the last day. Whole lots do not fit a hedge ratio, as the [stock futures pair lesson](/learn/path-18/module-18-8-pair-trading-with-stock-futures) showed for two legs, and ten legs round worse. A leg can enter the ban list. The statutory stack costs about the same for each leg round trip as for a pair, so a rule that turns half its names over every day pays it on every one of them. The F&O stock list also changes: SEBI revised the entry and exit tests in Aug 2024, so a basket drawn from today's list is not a basket of a year ago. Appendix: how the cutoffs were made The published trace cutoffs are for a very long sample. For a sample of a hundred and twenty five sessions the true cutoff is higher. I drew three thousand random walks of the exact length with a small drift, three series each, ran the same trace statistic on them, and took the ninety fifth percentile. As a check, the first half passed five and a quarter percent of baskets, and a long sample of five thousand sessions reproduced the published cutoffs to within about half a point. The numbers are a method of mine, not a standard. Watch in the terminal: does the app hold my money or shares? A short tutorial on sample data. ## The common mistake The claim: This basket of stocks passed a cointegration test, so the spread has to come back. A pass is a statement about one sample, at an error rate. Across many baskets, some pass by luck, and in this file those that passed in the first half did not pass again in the second. The weights of the basket moved with the window, and the factors that a rule strips out are in part noise below the edge. Before taking a basket seriously, count how many were screened, hold the weights fixed on days the search never saw, include statutory costs for every leg, and ask how many settings were tried. ## Key takeaways * Principal components strip out the shared market move, leaving each stock's residual. * Only factors above the noise edge are likely to be real. * The Johansen test asks how many stationary combinations a basket holds. * Baskets that passed a test once did not pass again more often than chance. * The weights of a basket can change sign between windows. * Count the baskets screened and the settings tried before trusting any pass. Two terms to hold on to from 18.13 From the module glossary. Source: Module 18.13, Baskets, PCA stat arb and Johansen tests. **Before you trade.** Derivatives trading carries a high risk of loss. SEBI's study "Profitability of Individual Traders in the Equity Derivatives Segment (FY25 to FY26)" (August 2026) found that 87.7% of individual traders in equity F&O made net losses in FY26, with aggregate net losses of Rs 91,685 crore. [Read the SEBI study](https://www.sebi.gov.in/reports-and-statistics/research/aug-2026/study-profitability-of-individual-traders-in-the-equity-derivatives-segment-fy25-fy26-_103835.html) What does Baskets, PCA stat arb and Johansen tests cover?Strip out the shared moves, test what is left, and count how many baskets you looked at. It works through The idea, plainly, Worked example: a year of Nifty 50 stocks, The common mistake, Key takeaways. It is module 18.13 of Path 18, Arbitrage and Relative Value, about 11 minutes of reading, free. TTry it ## Pair z score lab, on the same example. Hedge ratio, ADF test, half life and z score bands on two dated series. Dated NSE end of day data, nothing live, no orders. **Pair z score lab**. [Open the pair z score lab](/learn/tools/pair-lab) AF&O on RupeeCase ## Hedging only, in development F&O at RupeeCase will do one job: protect a portfolio you already own, by a rule published before it runs. No speculation, no naked option selling, no leveraged bets. It is not available yet. [How hedging will work](/hedging)[Sign up for RupeeCase](https://invest.rupeecase.com/signup) [No hedgeFalls with the marketWith a putA floor, for a premiumIllustration with made up numbers, not a product and not a recommendation.](/hedging) QQuick check ## 10 questions, 7 to pass. Each attempt draws 10 of the 15 questions in this module's bank, and a retake draws a fresh set. A pass gives this module's Certificate of Completion. Your result stays on this device and counts towards the Path 18 certificate. 7 of 10 correct passes module 18.13 Every answer is explained when you check; a retake is always free. Source: RupeeCase Learn quiz rules (70% to pass). Passed ### Certificate ID Download PNGPrint or save as PDF[Check the ID](/learn/certificates/verify) Self paced and unproctored. A record of completing free education by RupeeCase; not a NISM certificate and not endorsed by NISM, NSE, BSE or SEBI. It qualifies no one to deal, advise or publish research. CCheck yourself ## Path 18 so far. Your quick check results on this device, stop by stop. Stop 13 of 14 on Path 18 Arbitrage and Relative Value: every live stop, this module ringed. Source: RupeeCase Learn, Path 18. 1. [18.1Cash futures arbitrage and the Nifty 50 Arbitrage indexNot yet](/learn/path-18/module-18-1-cash-futures-arbitrage) 2. [18.2Conversions and reversalsNot yet](/learn/path-18/module-18-2-conversions-and-reversals) 3. [18.3Pair trading intuition: correlation is not cointegrationNot yet](/learn/path-18/module-18-3-pair-trading-intuition) 4. [18.4Futures calendar arbitrage and roll dynamicsNot yet](/learn/path-18/module-18-4-futures-calendar-arbitrage) 5. [18.5Box spreads with European index optionsNot yet](/learn/path-18/module-18-5-box-spreads) 6. [18.6Parity deviations in practice: what the frictions costNot yet](/learn/path-18/module-18-6-parity-deviations) 7. [18.7Cointegration toolkit: Engle Granger, ADF, half life and z scoreNot yet](/learn/path-18/module-18-7-cointegration-toolkit) 8. [18.8Pair trading with Indian stock futuresNot yet](/learn/path-18/module-18-8-pair-trading-with-stock-futures) 9. [18.9Mean reversion against momentumNot yet](/learn/path-18/module-18-9-mean-reversion-vs-momentum) 10. [18.10Implied correlationNot yet](/learn/path-18/module-18-10-implied-correlation) 11. [18.11Dispersion trades: construction and risksNot yet](/learn/path-18/module-18-11-dispersion-trades) 12. [18.12The correlation risk premiumNot yet](/learn/path-18/module-18-12-correlation-risk-premium) 13. [18.13Baskets, PCA stat arb and Johansen testsNot yet](/learn/path-18/module-18-13-pca-stat-arb#quiz) 14. [18.14Kalman filter hedge ratiosNot yet](/learn/path-18/module-18-14-kalman-filter-hedge-ratios) 15. [TPath test, 30 questionsNot yet](/learn/path-18/module-18-14-kalman-filter-hedge-ratios#pathTestSection) Path 18 in reading time: 143 minutes over 14 modules Minutes of reading per module, this one in amber. Source: RupeeCase Learn, Path 18. Explore next ## Put the module to work. * [ToolOption payoff builderLegs on a dated Nifty chain, breakevens and charges](/learn/tools/payoff-builder) * [Free module 18.7Cointegration toolkit: Engle Granger, ADF, half life and z score10 min read](/learn/path-18/module-18-7-cointegration-toolkit) * [Free module 18.8Pair trading with Indian stock futures10 min read](/learn/path-18/module-18-8-pair-trading-with-stock-futures) * [Free module 18.9Mean reversion against momentum10 min read](/learn/path-18/module-18-9-mean-reversion-vs-momentum) * [Free module 18.11Dispersion trades: construction and risks11 min read](/learn/path-18/module-18-11-dispersion-trades) * [Free module 18.14Kalman filter hedge ratios11 min read](/learn/path-18/module-18-14-kalman-filter-hedge-ratios) Community ## Join my WhatsApp group. Scan the code, fillFill one short form, and join my WhatsApp group. Slides, sources, factsheets and the rebalance logs live there, and members join a daily Google Meet with me. New to RupeeCase? [Getting started](/getting-started) [See the terminal](/terminal) [Sign up for RupeeCase](https://invest.rupeecase.com/signup) [Join the WhatsApp group](https://forms.gle/C6HMtM1Z2GfeocQV7) [One form, one group](https://forms.gle/C6HMtM1Z2GfeocQV7)Scan with your phone camera SSources ## Go to the source. 8 sources, NSE the most cited The publisher of each source listed below. Source: Module 18.13 source list. 1. [NSE, equity derivatives contract specifications (stock futures, lots, monthly expiry, delivery)](https://www.nseindia.com/static/products-services/equity-derivatives-contract-specifications) NSE 2. [NSE circular NSE/FA/73061, transaction charges from 1 Mar 2026](https://nsearchives.nseindia.com/content/circulars/FA73061.pdf) NSE 3. [NSE circular NSE/FATAX/73524, STT on equity derivatives from 1 Apr 2026](https://nsearchives.nseindia.com/content/circulars/FATAX73524.pdf) NSE 4. [SEBI circular of 30 Aug 2024, review of eligibility criteria for entry and exit of stocks in the derivatives segment](https://www.sebi.gov.in/legal/circulars/aug-2024/review-of-eligibility-criteria-for-entry-exit-of-stocks-in-derivatives-segment_86373.html) SEBI 5. [SEBI, broad framework for short selling: naked short selling is not permitted](https://www.sebi.gov.in/sebi_data/commondocs/ssframe_p.pdf) SEBI 6. [SEBI circular of 31 Dec 2018, physical settlement of stock derivatives](https://www.sebi.gov.in/legal/circulars/dec-2018/physical-settlement-of-stock-derivatives_41482.html) SEBI 7. [Avellaneda, M. and Lee, J.-H. (2010), Statistical arbitrage in the US equities market, Quantitative Finance](https://doi.org/10.1080/14697680903124632) Paper 8. [Johansen, S. (1991), Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models, Econometrica](https://doi.org/10.2307/2938278) Paper GTerms ## 5 terms from this module. Principal componentA direction of shared movement among stocks, found as an eigenvector of the correlation matrix of their returns. Also called a factor. Noise edgeThe largest eigenvalue that a correlation matrix of pure noise of the same size would give. Eigenvalues above it are likely real. ResidualWhat is left of a stock's return after the kept factors are removed. RankIn a Johansen test, the number of independent stationary combinations of the prices. Zero means no common home. Trace statisticThe Johansen statistic for a given rank, compared with a cutoff that depends on the sample length and the deterministic terms. [All 230 terms in the glossary](/learn/glossary) [Question people askWhat is statistical arbitrage?A short answer with a dated chart](/learn/questions/what-is-statistical-arbitrage)[Question people askCointegration or correlation: what is the difference?A short answer with a dated chart](/learn/questions/cointegration-vs-correlation)[Related module18.7 Cointegration toolkit: Engle Granger, ADF, half life and z score10 min, Advanced](/learn/path-18/module-18-7-cointegration-toolkit)[Related module18.8 Pair trading with Indian stock futures10 min, Advanced](/learn/path-18/module-18-8-pair-trading-with-stock-futures)[Pillar guideDerivatives and Quant115 modules across paths 12 to 19](/learn/derivatives) Education only, not investment advice, a research report or a recommendation to buy, sell or hold any security. RupeeCase (QC Alpha Technologies Pvt Ltd) is not registered with SEBI as an investment adviser or research analyst. Contracts and prices shown are for illustration only, come from NSE end of day files at least 30 days old, and are not recommendatory. Derivatives trading carries a high risk of loss.