Hi, for a project we used a single category implict association test (SC-IAT) similar to the one described by Karpinski et al. (2006)
Internal Consistency is an important topic for implicit measures. Thus and in an effort to keep it comparable to the original study we want to calculate the measure as reported by Karpinski et al. (2006), namely:
I' m reading out three steps for determining the internal consistency:
Now they write that they want to correct with the Spearman-Brown formula.
I never calculated the Spearman-Brown formula but found this:
According to this page (https://www.statology.org/spearman-brown-formula/) the Spearman-Brown formula is this: k * r / (1 + (k-1) * r)
Now “k: Factor by which the length of the test is changed.”. So in our case k = 3 / 1 = 3 or k = 72 (trials) / 24 (thirds) = 3.
Thus our “adjusted” (as Karpinski called it) reliability r would be =
r_adjusted = 3 * 0.44 / (1 + (3-1) * 0.44)
r_adjusted = 1.32 / (1 + 0.88) = .70
Is that correct? I am note sure if this is the proper way to calculate it.
Literature:
Karpinski, A., & Steinman, R. B. (2006). The Single Category Implicit Association Test as a measure of implicit social cognition. Journal of Personality and Social Psychology, 91(1), 16–32. https://doi.org/10.1037/0022-3514.91.1.16
Internal Consistency is an important topic for implicit measures. Thus and in an effort to keep it comparable to the original study we want to calculate the measure as reported by Karpinski et al. (2006), namely:
"To determine the reliability of the SC-IAT, we divided eachSC-IAT into thirds (blocks of 24 test trials) and calculated a SC-IAT score separately for each third of the trials without divid- ing by the standard deviation of correct response times. A measure of internal consistency was obtained by calculating the average intercorrelation among these scores. Dividing the task into thirds (or halves) underestimates the reliability of the entire measure. Fortunately, the Spearman–Brown correction can be applied to compensate for this underestimate of the true internal consistency for the entire measure (designated adjusted r; Nunnally, 1978). All internal consistency correlations reported in this article have been adjusted by using the Spearman–Brown correction. These adjusted reliability coefficients are conceptually equivalent and directly comparable to the Cronbach’s alphas."
- Split test trials (72) in thirds of 24 and calculate a score for the three thirds
- Build the average intercorrelation between the three scores
- Correct with Spearman-Brown formula.
Now they write that they want to correct with the Spearman-Brown formula.
I never calculated the Spearman-Brown formula but found this:
According to this page (https://www.statology.org/spearman-brown-formula/) the Spearman-Brown formula is this: k * r / (1 + (k-1) * r)
Now “k: Factor by which the length of the test is changed.”. So in our case k = 3 / 1 = 3 or k = 72 (trials) / 24 (thirds) = 3.
Thus our “adjusted” (as Karpinski called it) reliability r would be =
r_adjusted = 3 * 0.44 / (1 + (3-1) * 0.44)
r_adjusted = 1.32 / (1 + 0.88) = .70
Is that correct? I am note sure if this is the proper way to calculate it.
Literature:
Karpinski, A., & Steinman, R. B. (2006). The Single Category Implicit Association Test as a measure of implicit social cognition. Journal of Personality and Social Psychology, 91(1), 16–32. https://doi.org/10.1037/0022-3514.91.1.16
Last edited: