Pilot Study Numbers

#1
What is an appropriate number of responses for a pilot study? I'm trying to find something in the literature to support the size. Specifically, since Cronbach's alpha is being calculated from the pilot study data - is there a minimum? Is there a formula similar to sample size?

I sent out 128 surveys, 12 responded to date.
 
#2
Hello,
I think you can calculate standard error of Cronbach's alpha, but the problem is I cannot think any step further to justify the size unless you have an idea how precise you want your estimation from your pilot study to be.
Unless you are doing something really statistic, I guess you only need to prove your Cronbach's alpha of the final sample is okay. It might be interesting, but it might not be really critical to justify the pilot sample size itself mathematically. Instead, it might be more practical to justify the representativeness of the pilot sample, to ensure that your final Cronbach's alpha is not deviated far from your initial estimate. Certainly I would say sample size will still be something to consider as far as representativeness is concerned, and you might need to sample size at least to show a distribution of data. Therefore I would say you are aiming a 30ish sample before you are actually able to observe what is going on with the scales.
But as I say it is interesting, so hopefully there will be statistic people to answer this question directly.
Bests,
 
#3
After a week, and 1 additional prompt, 17 responses are available for analysis. These were put into SPSS and Cronbach's alpha were run. Values between .8 and .96 were returned by the software.

A panel of teachers reviewed the survey instrument with me and we rearranged the headings, clarified some of the statements, added a statement, and fixed a spelling error.

In reading about pilot studies, procedures are the main focus. Is there anything else that I should do before proceeding with the data collection from other school districts? For example, is it possible to determine sample size from the returned surveys from the pilot study?
 
#4
Hello,
If you imagine that you will have some statistical tests (not just descriptives) in the end, you might want to estimate the target sample size. Try google "power calculation."
 
#5
Hello,
If you imagine that you will have some statistical tests (not just descriptives) in the end, you might want to estimate the target sample size. Try google "power calculation."
Thank you. I was unsure about the keywords to use to find information. Besides SPSS SamplePower, two free programs were found:

PS

G*Power

However, my understanding is too rudimentary and looking through stats books is just adding to the confusion. What I really want to know is:

(1) The justification of a small response rate on a pilot study for continuing with the actual study. Several studies in the literature (e.g., one using the original survey I have modified) use only 15-17 responses in the pilot study. No dissertations that I have found even address if this number is sufficient to really toss out a question or add another one.

(2) Sample size calculations from data collected during a pilot study. No dissertation I have found discusses how they arrived at sample size from a pilot study. No articles in education appear to even bother with this but my understanding is that a poor sample size can make the study a waste or lead to incorrect assertions.

-- Now, I want to do the proper things before running the actual study because after is just fudging. I've noticed, though, there is no - do this - do this - do this or that - style of flow charts available for a novice like me. Instead there are just tons of buzzwords tossed around. Maybe the categories are too great for such a chart. However, isn't this why Stevens even created nominal, ordinal, interval, and ratio categories - to make things more clear for the researcher?

Here is my list. After running a pilot study the survey researcher should:

(1) run Cronbach's alpha
(2) run power calculations to determine sample size
* adjust number of surveys to be sent
* Increase if too low
* Decrease if too high
(4) run sample though tests such as ANOVA, etc that will be run on data

If this is the list then I am stuck on number 2 :rolleyes:

PS. As always, I'm thankful for this site to see others asking questions. Statisticians certainly have their own language - and they appear to love it that way. :yup: