single-case study

[collegeblock]

Dear friends,

I am running a single case study. I am collecting correct and incorrect responses on two similar tests in 3 moments in time. I would like to know two things:

- if there exist significant differences between each of the three times within each test and where are those differences.

- if there exist differences between the tests in each of the points in time.

I have read I can use McNemar test because I am working with dichotomous nominal data. Despite that, I am working with three moments in time and not two. Should I use a chi-square instead? I also read some stuff on trend analysis but I did not quite that.

Any help is appreciated.

 

[collegeblock]

I think I am finding the answer. I need to use the Cochran's Q test. If anyone is sure about it, please let me know.

http://www.vsni.co.uk/products/genstat/htmlhelp/npt/CochranQ.htm

 

[collegeblock]

I am reading I can run a COCHRAN'S Q test. Is that suitable in these design, bearing in mind I am working with only one participant?

 

[trinker]

I don't think you can use anything with a single case other than simple differences in scores. Am I correct you have 3 scores for one observation? I think you may need to provide more information so we know what your data looks like.

 

[collegeblock]

Thanks for the answer Tinker.

This is the scenario:

I am working with 2 tests and I do 3 observations for each test. Each test has a 100 items out of which I collect the number of correct responses and the reaction times for each response. In the first test the participant sees a 100 pictures and names them. I measure whether the answer is right or wrong and the time the participant takes to answer each item once the picture is on the screen. In the second test, the participant also names the pictures but they have other names.

Considering the error count and the reaction times, I want to see:
(1) if in the first test there are differences between the first, the second, and the third time observation.
(2) if in the second test there are differences between the first, the second, and the third observation.
(4) if there are differences between the two tests for each of the observations.

For the error count, I was told I can look at the percentages of right responses and then see if there are differences in those. This is why I came about with the COCHRAN'S Q test, even though I am not so sure whether this is the best test.
My idea is that the COCHRAN'S Q test will tell me if there are differences in the tests and then I can go do post-hoc tests using the McNemar?


For the reaction times I am not sure what to use because the data is not dichotomous.

 

[collegeblock]

single-case study (useful link)

Hello! I have been burning my eyebrows reading here and there so to find ways to work out proper statistical analysis on data coming from one participant. I found chi-square tests are useful all over (e.g. McNemar, Cochran's Q test).

I also found some scholars who do not use these methods properly and refer to the book by Siegel & Castellan (1988, Non parametric statistics for the behavioral sciences) which, otherwise, is been so far a good reading.

After rambling around here and there, I ended up stumbling on these website from the University of Aberdeen which sort of cleared up some quite good stuff for me: http://www.abdn.ac.uk/~psy086/dept/SingleCaseMethodology.htm

Hope that helps!

 

[hlsmith]

CollegeBlock,
You are definitely doing something uncommon, so finding a right fit statistic that your data meets the assumptions for will be very difficult. First issue and probably the biggest hurdle is that you only have one person. In general, your research sample should be as representative of the population as a whole as possible (random selection is used at times to attempt to procure such a sample). However, having trust that your one person represents every person who would take these tests is pretty much impossible. Throwing this hurdle out the window and ignoring it for now, perhaps you can test the intra-reliability of the tests. Perform a Kappa test on the three tests per each of the two test groups. These data will tell you whether upon retest a subject gets the same results. You may then, use the two test kappa means to compare to each other to see differences in intra-reliability between the two tests (this is probably breaking most rules and probably would not hold up at all under peer-review, but could allow you to compare the two different tests). Also if these tests are diagnostic or are supposed to classify a person – do you have a true gold standard for classifying the person, if so you may be able to explore something in the area of cross classification tables. Though, all of this is a real big stretch from my perspective, since you are just examining a single person with their result not being generalizable to anyone else. Somebody out there may come up with something better than this suggestion.

HS

P.S., Does the subject get better everytime they take the test, if so you need to control for this or quantify it, since if it occurs differently between the two test it could bias conclusions.

 

[collegeblock]

Thanks hlsmith,

I am reading in those links I posted that it is indeed possible to do science with single case studies. I agree with the fact, though, that the smaller the population the more difficult it is that it will represent overall population. In any case knowing how does an individual patient do is relevant for the patient itself and thus for clinical practice as well.

On the links by Uni Aberdeen I posted, I read that one can go and do:
- Fully standardized neuropsychological tests and then compare the performance of the patient with the normative data.

- Do what they call 'intra-individual' comparisons (with chi-square tests; here is where my first question with the mcnemar and the cochran's q test came about)

- and, then compare a patient's performance to a (modestly sized) matched control sample.

In my case, I might be able to compare my data with a fully standardized neuropsychological tests since the items of my test are actually being standardized for that matter. In relation using this method, the guys from Uni Aberdeen seem to have some kind of computer programs that run the statistics for you. Those programs, though, require you to compute a mean and standard deviation. Since I am working with dichotomous data (basically rights and wrongs) I am wondering how to compute the mean and standard deviation for those.

Should I compute the mean and multiply it by a 100 ??

 

[collegeblock]

Forget about the means with the dichotomous data. It was a stupid question You cannot compute means on dichotomous data.

 

[collegeblock]

This is what I am planning on doing: I am running a single case study with one participant. The goal is to see whether test A provides additional information than test B. I would like to know whether the following quantitative analysis seems reasonable. Any comments are more than welcome!

- 2 measures (test A and test B)
o The items of each test are controlled for imageability, age of acquisition, log frequency…
o Both tests are standardized.
- Continuous data (Reaction Times) and Dichotomous data (Error count)
- 3 moments of assessment
- I want to see
o differences in Reaction Times between tests & differences in Reaction Times between the 3 moments of assessment within each test
o differences in Error Count between tests & differences in Error Count between the 3 moments of assessment within each test.
- 60 items in each test
- 1 participant

On Reaction Times I am planning to conduct
- An item based analysis
o Mixed ANOVA? (there is 60 measurement points per test and assessment)
- A multiple regression analysis (stepwise method) to see what is the effect of imageability, age of acquisition, log frequency…
 On the Error Count I am planning to conduct
- To test the difference between the individual scores in the 3 moments of assessment time with the scores of the normative sample (SINGLIMS.EXE). I will do that for each test and for each moment of assessment.
- To test the difference between tests (DISSOCS.EXE). I will do that for each moment of assessment.
- To test the difference between the 3 moments of assessment within each test (
o COCHRAN’S Q TEST to see if there are differences and afterward
o McNemar tests to see where are the differences
- A multiple regression analysis (stepwise method) to see what is the effect of imageability, age of acquisition, log frequency…