Is the correlation reliable?

[zealot]

Hello!!!

I hope you can help me with this question. I have a set of daily price data for six months for two commodities. I calculated the correlation between these two commodities for the whole data set and it shows to be very low, -0,11 (C & D in the attached sheet). I need to determine if this correlation is valid from month to month. After I have calculated correlation for each of the months they proved to be very different. I need to establish if the 6 month correlation is valid by looking at the monthly correlations (I tried to find the standard deviation of the monthly correlation coefficients and it certainly looks promising).

I made the same month to month breakdown for another set of daily prices of 6 month for a different pair of commodities (A & B in the attached sheet) and it showed that the correlation was very high – namely, 0,91. I calculated the standard deviation between the correlations for each of the months and it showed to be quite low – 0,16. Do you think calculating the standard deviation of the sub periods of a set of data can help to prove the stability of the correlation of the whole data set, that the correlation is systematic? I am new to the statistics and would love to hear any kind of feedback.)

Thanks a lot in advance,

Dima

I included the excel sheet with all the calculations.

 

[zealot]

no ideas at all? ((

 

[JohnM]

I don't believe there is any correlation at all between C and D - the 6-month correlation is nonexistent, and the month-to-month variation appears to just be a random fluctuation.

In terms of A and B, there appears to be a correlation that is dependent on some factor of time (seasonality, etc) - stronger in some months, weaker in others. The key is to find out why and see if it holds up over time. Six months may not be long enough....

The size of the month-to-month standard deviations don't prove or disprove anything - it may be interesting, but the real key to finding and using a good, reliable correlation is to figure out why the correlation exists in the first place - a reason outside of the data itself.

The reason probably lies in the processes by which people set the prices of the commodities.

 

[zealot]

Hello John,

Thanks a lot for your reply….Actually here is the only forum where I was able to get more or less clear feedback on my question so pleas do not disappear…)) I am trying to find if there is a relation between the variance of the sub-period correlations an dthe correlation of the whole period becaue I need to find a method for establishing the significance of the correlation. I got this sub-period idea from the book on money management – “Money Management Strategies for Futures Traders”:

A good way of judging whether a correlation is genuine or otherwise is
to rework the correlations over smaller subsample periods. For example,
the period 1983-1988 may be broken down into subperiods, such as
1983-84, 1985-86, and 1987-88, and correlations obtained for each of
these subperiods, to check for consistency of the results. Appendix C
presents correlations over each of the three subperiods.
If the numbers are fairly consistent over each of the subperiods, we
can conclude that the correlations are genuine. Alternatively, if the numbers
differ substantially over time, we have reason to doubt the results.
This process is likely to filter away any chance relationships, because
there is little likelihood of a chance relationship persisting with a high
correlation score across time.
Table 4.7 illustrates this by first reporting all positive correlations
in excess of t-O.80 for the entire 1983-88 period and then reporting
the corresponding numbers for the 1983-84, 1985-86, and 1987-88
subperiods.
Table 4.7 reveals the tenuous nature of some of the correlations. For
example, the correlation between soybean oil and Kansas wheat is 0.876
between 1987 and 1988, whereas it is only 0.410 between 1983 and
1984. Similarly, the correlation between corn and crude oil ranges from
a low of -0.423 in 1987-88 to a high of 0.735 between 1983 and
1984. Perhaps more revealing is the correlation between the S&P 500
and the Japanese yen, ranging from a low of -0.644 to a high of 0.949!
Obviously it would not make sense to attach too much significance to
high positive or negative correlation numbers in any one period, unless
the strength of the correlations persists across time.
If the high correlations do not persist over time, these commodities
ought not to be thought of as being interrelated for purposes of diversification.
Therefore, a trader should not have any qualms about buying
(or selling) corn and crude oil simultaneously. Only those commodities
that display a consistently high degree of positive correlation should be
treated as being alike and ought not to be bought (or sold) simultaneously


I included the picture of the table that the author was referring to. Hope you can comment on using this method of establishing the significance of the correlation. Finding the variance of the subperiod correlations is just my way of “mathematizing” the authors words that “Obviously it would not make sense to attach too much significance to high positive or negative correlation numbers in any one period, unless the strength of the correlations persists across time”

Hoping for any kind of feedback)

Dima

 

[JohnM]

I think the approach has merit and makes sense, but make sure it is used with a healthy dose of reality and practical judgment - do not rely on correlations and variances alone.

Maybe it could be used as a way to rank-order possible pairings or groupings of commodities, in a relative sense, rather than an absolute judgment of the actual correlations....

 

[zealot]

yep...thanks a lot for your feedback!!!...I am going to use the subperiod variance as confidence factor in establishing the significance of the whole-period correlation...

The original reason is to find pairs of commodities which are either reliably highly or lowly correlated....by the way....do you think that the same method can be used to prove that systematic correlation does not exist?...i.e when the sup-period correlations are highly varied...

Thanks again!!!!

Dima