Help with experience-sampling data analysis plan and multilevel linear modelling

#1
Hi everyone!

I'm conducting a research project for my thesis, and I'm hopelessly lost.

My aim is to see whether gaming enjoyment (liking) decreases during a gaming session, and then to see whether that decrease is affected by scores on a gaming disorder test (GDT). Each participant will have approximately four scores for liking, each collected an hour apart during the gaming session. They will also each have one score for the GDT.

From what I have read, multilevel linear modelling (MLM) seems to be the way to go, but I'm really struggling to understand it. Most examples of MLM involve taking the mean of the momentary data for each participant - in this case, that would be the mean of the liking scores. But I'm not interested in their average liking during the session, but rather the rate of change in liking over the course of the session, and then seeing if this differs between participants based on their GDT scores.

Any help here would be hugely appreciated!

Thanks,

Mike
 

AngleWyrm

Active Member
#2
Hi everyone!
My aim is to see whether gaming enjoyment (liking) decreases during a gaming session

But I'm not interested in their average liking during the session, but rather the rate of change in liking over the course of the session
So it's a measurement that can change over time, and you're interested in the rate of change, aka acceleration.

enjoyment / sessionTime^0 = location
enjoyment / sessionTime^1 = velocity
enjoyment / sessionTime^2 = acceleration
 

katxt

Well-Known Member
#3
One way would be to convert each subject's liking scores into one rate of change score by effectively finding the slope of the liking vs. time graph for each subject. Then look for a significant correlation or regression between rate and GDT
 

AngleWyrm

Active Member
#4
Example data, with polynomial trend line for x
starting attitude = x^0 = 0.875
velocity = x^1 = 0.196x
rate of change (acceleration) = x^2 = 0.161x^2
Untitled.png
 
#6
Thanks everyone for your replies so far!

My overall research question is: what is the relationship between scores on a gaming disorder test (GDT) and gaming enjoyment?

I'm wondering now if I can answer that more simply by:

1/ assigning each participant one liking score based on the average of their in-game repeated-measures liking scores
2/ splitting participants into two groups (high and low GDT scores)
3/ comparing the liking scores of the groups

One potential problem with this is that some participants will game for longer than others, and so will have more in-game datapoints for liking (as they are asked every hour how much they are enjoying gaming). I'm not sure if this invalidates this method, and I'm better off returning to calculating the rate of change in liking rather than average liking.

Any help much appreciated!

Mike
 
#7
One way would be to convert each subject's liking scores into one rate of change score by effectively finding the slope of the liking vs. time graph for each subject. Then look for a significant correlation or regression between rate and GDT
Yes, that's a good idea, and I did think of that. My hypothesis is that there won't be a correlation between GDT scores and individual rates of change in liking, so I think I can test this by splitting participants into two groups (high and low GDT) as described above and then comparing rates of change. I think so, anyway - please correct me if I'm wrong!
 

katxt

Well-Known Member
#8
What you have suggested is a low power way of testing by regression. I would start with plotting change against GDT. If there is any noticeable pattern then go ahead with the regression. If not, go ahead anyway and get a high p value. then you can say that there is no good evidence for a connection.
 

AngleWyrm

Active Member
#9
One potential problem with this is that some participants will game for longer than others, and so will have more in-game data points for liking (as they are asked every hour how much they are enjoying gaming). I'm not sure if this invalidates this method
The problem you're referring to is standard error = (standard deviation) / Sqrt(sample size), used for z-score = (observation - mean)/(standard error) that measures how outstanding an individual is relative to the sample average. As the sample size goes up the standard error goes down, and so too the z-score ranking an observation against the sample population.
 

AngleWyrm

Active Member
#11
Standard error solves problems that standard deviation alone can't.

Standard deviation -- average distance away from average -- is good for some problems, but doesn't have anything to say about sample size. Whereas standard error takes into account the sample size; it's still standard deviation, but scaled by a function of sample size, sqrt(n). The result is the bell
curve becomes narrower as sample size goes up, and wider as sample size goes down.

In the video, the author uses a look-up table. This online calculator provides the same information.
 
Last edited:

Karabiner

TS Contributor
#12
I'm conducting a research project for my thesis, and I'm hopelessly lost.
My aim is to see whether gaming enjoyment (liking) decreases during a gaming session, and then to see whether that decrease is affected by scores on a gaming disorder test (GDT). Each participant will have approximately four scores for liking, each collected an hour apart during the gaming session. They will also each have one score for the GDT.
It would maybe be helpful if you would further specify your research question and your theoretical assumptions.

You could calculate the difference of the liking score between first and last observation and use this as dependent
variable (DV).
You could calculate the difference between first and minimum liking score as DV.
You could define a theoretically meaningful observation period for all subjects (e.g. 3 hours) and calculate the
difference between initial liking and liking at the end of this period; or you could do this repeatedly (something
like 2, 4, 6 hours, with last observation carried forward for earlier drop-outs).
You could regress the liking score on time and use the regression coefficient as DV. One could contemplate
how to account for possible non-linear relationships (e.g. U-shaped or reverse J-shaped etc.)
You could use MLM as mentioned in your title, and regress liking score on time, adding GDT as predictor.

By the way, do not split into high and low GDT.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1458573/pdf/bmj33201080.pdf
https://discourse.datamethods.org/t/categorizing-continuous-variables/3402
You can use the original value(s) for whatever you want to do.

With kind regards

Karabiner