Degrees of freedom


Yesterday I did some backtesting of algorithmic trading systems to study 4-rule (n = 67) versus 2-rule (n = 65) strategies. I ran five two-sample t-tests assuming unequal variances (in Excel) to compare profit/loss (PNL), PNL/max drawdown, # trades, average trade, and profit factor.

Across the five tests, degrees of freedom varied from 76 to 125. I thought df was a defined number based on sample size, but obviously I'm wrong. Why would this vary?

They are related in that I compared the same two groups along the lines of five different variables.

That makes sense if df is related to pooled variance. PNLDD is a number between -1 and 7 or so whereas PNL is between +/- 5000.



Active Member
Hi Mark,

You may look at multiple comparisons, to think if you need to take a smaller significance level.

As you can see it is the opposite if it is pooled variance the DF calculation is simple.
But for unequal variances, it is more complex (pooled DF)


Active Member
If I were using Bonferroni correction, could I just divide 0.05 by five for five comparisons?

Hi Mark,

Correct, but the Bonferroni is not accurate better use the more accurate, The Sidak
But the difference is minor 0.010206 instead of 0.01

The problem of both calculations, that it protects against type 1 error at the expense of increasing type 2 error.
it assumes total independence which is usually not the case.

The Holm correction is a bit more balanced.
Last edited: