Is this a type of cross-validation?

#1
Hi

I did the following procedure for my paper:
We needed hourly weather data for all years. But, we had only for some years this data. For most years we had 3-hour data. Therefore, I needed to interpolate 3-hour data to make hourly ones.
But I should have checked the accuracy of the interpolation method. Therfore, I did this procedure:

I had a set of hourly weather data.
I eliminated 2/3 of the data (two points in each 3 hours) and made a 3-hour data. (I made Artificial gaps)
Then, I interpolated this set in order to make an hourly set.
Finally, I checked the accuracy of this interpolation by RMSE.

Is this cross validation?
Thanks
 

hlsmith

Not a robit
#2
This is closer to making data splits. If you made models for each fold and applied to hold out sets that would be closer to CV. Nothing wrong with the approach.

How did you select which points to remove? What is your sample size?
 
#3
This is closer to making data splits. If you made models for each fold and applied to hold out sets that would be closer to CV. Nothing wrong with the approach.

How did you select which points to remove? What is your sample size?
The fact is that I removed regularly. For instance, I retained hour 1, and omitted hour 2 and 3. And then I retained hour 4, and so on. Then,I interpolated the data and estimated te values for hours 2 and 3.
 

hlsmith

Not a robit
#4
What is the purpose of this model?

Yeah, as mentioned this could be called non-random data splits, creating a training and test set.
 
#5
What is the purpose of this model?.
I need hourly weather data of 16 years. (The size of each year's data set is 8760.)
I have hourly data for three years. For the remaining 13 I have 3 hour data. Therefore, I should interpolate the data to make hourly ones.
I need to check if the interpolation method is OK. Therefore, using the three years' hourly data available, I did the mentioned procedure.

Actually, I did this procedure for my paper, and the reviewer asked me whether it is cross-validation or not. Honestly, I did not understood your answer because my major is not statistics. Could you please explain more?
 
#6
This is closer to making data splits. If you made models for each fold and applied to hold out sets that would be closer to CV. Nothing wrong with the approach.

How did you select which points to remove? What is your sample size?
Let me explain what I understood from your answer to check whether I got the right thing.

I have three years of hourly data which are training sets.
After making artificial regular gaps, I have three years of 3-hour data which are my test sets.
I check my interpolation methods by testing on these data. However, in CV the gaps should be random, and not regular. Therefor, it is not CV, but it is data splitting.

Right?
 

hlsmith

Not a robit
#7
Crossvalidation you would fit a model to each set and score the other set. So you would fit a model to 1,2 data then score 3 hour set, then fit model to 3 hour dataset and score 1,2 hour data.
 
#8
Crossvalidation you would fit a model to each set and score the other set. So you would fit a model to 1,2 data then score 3 hour set, then fit model to 3 hour dataset and score 1,2 hour data.

So, my procedure is not any type of cross validation. Am I right?
 
#10
You have a validation set, but not doing the cross part.
Thank you for the time you allot here. Would you please say your opinion about this response to my reviewer:

In fact, our procedure is a type of validation with training and test sets. However,it is a little different from cross-validation, in which the original sample is divided into multiple subsamples, and the model is tested with different training and test sets. However, one type of validation which is sometimes called a cross validation is near to our procedure; this is holdout method. In holdout method, the data is divided into two sets randomly,and one single run is enough for validation. The fact is that our procedure is different from this method too because our data is regularly divided into two parts.
 

hlsmith

Not a robit
#11
Is that what you all are writing back to a reviewer? This site allows you to Personal Message me (by clicking on my profile). Go ahead and send me exactly what the reviewer wrote and I can provide my comments/opinions.
 
#12
Is that what you all are writing back to a reviewer? This site allows you to Personal Message me (by clicking on my profile). Go ahead and send me exactly what the reviewer wrote and I can provide my comments/opinions.
Well, actually I want to learn and respond to the reviewer
 

hlsmith

Not a robit
#13
It is unclear to me if your prior block of text in post #10 was you all's rebuttal to a reviewer or their comments. If we don't know what the reviewer's issues were it is hard to understand where you are coming from or what issue you are attempting to address.
 
#14
It is unclear to me if your prior block of text in post #10 was you all's rebuttal to a reviewer or their comments. If we don't know what the reviewer's issues were it is hard to understand where you are coming from or what issue you are attempting to address.
Thank you hlsmith.
I will send you a personal message.