Spearman-Brown (Prophecy) Formula - Internal Consistency in SC-IAT

#1
Hi, for a project we used a single category implict association test (SC-IAT) similar to the one described by Karpinski et al. (2006)
Internal Consistency is an important topic for implicit measures. Thus and in an effort to keep it comparable to the original study we want to calculate the measure as reported by Karpinski et al. (2006), namely:
"To determine the reliability of the SC-IAT, we divided eachSC-IAT into thirds (blocks of 24 test trials) and calculated a SC-IAT score separately for each third of the trials without divid- ing by the standard deviation of correct response times. A measure of internal consistency was obtained by calculating the average intercorrelation among these scores. Dividing the task into thirds (or halves) underestimates the reliability of the entire measure. Fortunately, the Spearman–Brown correction can be applied to compensate for this underestimate of the true internal consistency for the entire measure (designated adjusted r; Nunnally, 1978). All internal consistency correlations reported in this article have been adjusted by using the Spearman–Brown correction. These adjusted reliability coefficients are conceptually equivalent and directly comparable to the Cronbach’s alphas."
I' m reading out three steps for determining the internal consistency:
  1. Split test trials (72) in thirds of 24 and calculate a score for the three thirds
  2. Build the average intercorrelation between the three scores
  3. Correct with Spearman-Brown formula.
From my colleague I now got a matrix with the dimension: 60x3. We got the data of 60 participants (60 rows) and the 72 test trials (d-scores) where split into thirds of 24 trials and those were used to build a d-score (as recommended above), thus the matrix contains three columns with three d-scores for the three thirds of the test trials. Now from what I understand they calculated the average intercorrelation. Thus to replicate I calculated three Spearman correlations between the first-second, second-third, first-third. And build an average of those, resulting in an average intercorrelcation of r = . 44.

Now they write that they want to correct with the Spearman-Brown formula.
I never calculated the Spearman-Brown formula but found this:

According to this page (https://www.statology.org/spearman-brown-formula/) the Spearman-Brown formula is this: k * r / (1 + (k-1) * r)

Now “k: Factor by which the length of the test is changed.”. So in our case k = 3 / 1 = 3 or k = 72 (trials) / 24 (thirds) = 3.

Thus our “adjusted” (as Karpinski called it) reliability r would be =

r_adjusted = 3 * 0.44 / (1 + (3-1) * 0.44)

r_adjusted = 1.32 / (1 + 0.88) = .70

Is that correct? I am note sure if this is the proper way to calculate it.

Literature:

Karpinski, A., & Steinman, R. B. (2006). The Single Category Implicit Association Test as a measure of implicit social cognition. Journal of Personality and Social Psychology, 91(1), 16–32. https://doi.org/10.1037/0022-3514.91.1.16
 
Last edited: