Item Response Theory vs Confirmatory Factor Analysis

#1
Hi all,

I was wondering what the core, meaningful differences are between Item Response Theory and Confirmatory Factor Analysis.

I understand that there are differences in the calculations (focusing more on item vs. covariances; log-linear vs. linear).

However, I have no idea what this means from a higher-level perspective - does this mean that IRT is better than CFA in some circumstances? Or for slightly different end-purposes?

Any musings would be useful as a scan of the research literature led to more a description of IRT and CFA than any useful comparison of the core differences between them

Simon.
 

noetsi

Fortran must die
#2
Its been several years since I worked with IRT, but one obvious comment is that CFA is a method/tool (one that I think can be used for IRT) and IRT is a theory or set of theories about measurement. While you can use IRT techniques to generate calculations, comparing CFA to IRT is sort of like comparing theoretical physics to a telescope. There are many many different methods within IRT (notably tied to how many parameters are estimated) but behind it is a theory and philosophy totally lacking in CFA.

Are you asking about say comparing a one parameter IRT model to CFA used to measure the same element?
 

VPG

New Member
#3
Actually, IRT and CFA are very closely related. IRT, while called a theory, is actually just a collection of latent variable models that attempt to model the relationship between item responses and a latent variable (much like CFA). The primary difference, as you mentioned, is the parametrization of the models. However, in many cases parameters from one model (categorical CFA using dichotomous items) can be converted into an asymptotically equivalent IRT model (2 parameter logistic model).

An important difference between CCFA (categorical CFA) and IRT is the estimation of model parameters. The different estimation techniques have lead to the different models being better at different things. For example, testing the number of dimensions was much easier in FA (historically) while scoring was much more developed in IRT. As estimation and computing continues to advance, the differences between these modeling frameworks is disappearing.

As psychometricians we often switch back and forth between the two modeling traditions depending on what we're doing at the time. To make life easier - you may want to just think of both techniques as "item factor analysis" (assuming you are using CFA models appropriate for item level data). There is a 2007 paper by Wirth & Edwards on IFA in the journal Psychological Methods. You may find it interesting.
 

Dragan

Super Moderator
#4
Hi all,

I was wondering what the core, meaningful differences are between Item Response Theory and Confirmatory Factor Analysis.

I understand that there are differences in the calculations (focusing more on item vs. covariances; log-linear vs. linear).

However, I have no idea what this means from a higher-level perspective - does this mean that IRT is better than CFA in some circumstances? Or for slightly different end-purposes?

Any musings would be useful as a scan of the research literature led to more a description of IRT and CFA than any useful comparison of the core differences between them

Simon.
I'll weigh in and list some differences (albeit some have been already mentioned)

1. FA is a data reduction technique whereas IRT focuses on accurately modeling the interaction between persons and items.

2. FA focuses on the correlation or variance/covariance matrices while ignoring other item characteristics, such as the mean or standard deviation (SD) of the variables. MIRT, however, does not use standardized variables and instead uses the mean and SD of the items as represented by item difficulty and discrimination.

3. A third difference lies in marginalization and how model parameters are estimated.

4. In IRT, efforts have been taken to use the same latent space across tests and persons to keep a common scale for all analyses. This does not receive much attention in a FA approach.
 

CB

Super Moderator
#5
I think the new poster VPG is spot on here. CFA and IRT are actually very closely related. Both attempt to estimate the relationships between continuous latent variables and responses to observed indicators. Some IRT models (e.g., 1PL, 2PL) are formally equivalent to related CFA models.

noetsi raises the issue of that word "theory", but I think that term is a bit of a misnomer in the term Item Response Theory. The "theory" only becomes specific and testable when we choose a specific model to represent the relationship between the trait(s) and the items. This is exactly the same case as with CFA: CFA itself is not a testable theory, but an actual specified CFA model is. Don't be confused by the terms. Neither IRT nor CFA are theories, but one can specify IRT or CFA models that are theories (or that encapsulate theories).

Some apparent differences between the two are just issues of tradition. E.g., Dragan talks about FA usually being based on variance/covariance matrices, while IRT uses mean and SD. But it's obviously possible to model means and standard deviations in CFA too.

does this mean that IRT is better than CFA in some circumstances? Or for slightly different end-purposes?
I'd say the latter. CFA is a general tool for looking at the relationship between latent variables and observed indicators. IRT can be seen this way too, but in general users of IRT are interested in a very specific domain of questions (i.e., relationships between latent variables and test items, with interests in things like comparing persons who have completed different items, maximising reliability while minimising the number of items completed, computerized adaptive testing, etc.) So depending on what your research question is, you might find that one of the two traditions has more useful tools and a more developed literature relating to the job at hand.