Moderation Analysis

#1
Hi,

I am conducting research on the relationship between certain personality traits (Personal Need for Structure; PNS) and Creativity. I conducted an experiment whereby one group of participants were given 'reassuring' instructions to a creativity task and others just standard instructions (control). After the activity, participants were asked to report their 'fear of being incorrect' using a 7 point likert-like scale (let's call that variable 'sense of reassurance').

So... my analysis to date has showed that
- There is a significant difference in reported sense of reassurance between the experiment and control group (t-Test)
- There is no significant difference in creative performance between experiment and control group (t-Test)
- Moderation analysis showed that GROUP does NOT moderate the relationship between PNS and Creativity (which I expected)
- Moderation analysis shows that SENSE OF REASSURANCE DOES moderate the relationship between PNS and Creativity. Simple slopes shows that when PNS is low (only) that the moderation is significant.

What I don't understand is why using Group is the moderator results in a non-sig result but when sense of reassruance is used, the result is significant (given that sense of reassurance is the only observable difference between the two groups (checked everything else like gender, age, other personalities traits etc and all appear to be non-sig - so groups appear to be comparable).

My only theory so far is that when 'Group' is used as a moderator, it only gives two levels to conduct the regression (experiment and control) whereas when the sense of reassurance variable is used, it gives 7 levels (7 point likert-like scale). Is there any merit in this idea that moderation is more sensitive when a ratio variable is used rather than a dichotomous categorical variable?

Thanks in advance,
Jenny
 
#2
After the activity, participants were asked to report their 'fear of being incorrect' using a 7 point likert-like scale
Do you mean a single Likert-type item, or indeed a Likert scale (which consists of several Likert-type items)?

Another problem: You ask about difference between results, but you did not report them properly, e.g.
What I don't understand is why using Group is the moderator results in a non-sig result
This is uninformative - p=0.05? p=0.99? And you should preferably report sample sizes , regression weights, standard errors in addition.
but when sense of reassruance is used, the result is significant
p=0.0499? p=0.0000001?
There is a significant difference in reported sense of reassurance between the experiment and control group (t-Test)
How large was the difference (group means, SD)? What was the p-value?

With kind regards

Karabiner
 
#3
Thanks Karabiner for taking the interest in my question! I guess my initial thought was if it was theoretically valid re a difference between categorical vs scale variable as moderator. That said... here are the stats that may assist you understand my particular example better...

APA now refers to Likert as 'Likert-like' but it was simply a Likert scale. The question asked participants if they felt worrisome thoughts of being incorrect during the task (1 = strongly disagree, 6 = strongly agree). A lower score indicates a greater sense of reassurance.

Overall sample (n = 116)
PNS - M = 4.11, SD = .98
Creativity - M = 2.37, SD = .24
Sense of reassurance - M = 3.34, SD = 2.06
Note - Sense of reassurance has non-normal distribution - positive skew however when running assumptions prior to regression outliers and influential cases etc were within acceptable range. Cooks distance max = .069, Mahalanobis max = 8.205, VIF = 1.017, Tolerence = .984.

For experiment group (n = 60)
PNS - M = 4.18, SD = .97
Creativity - M = 2.38, SD = .25
Sense of reassurance - M = 2.97, SD = 1.97

Control group (n = 56)
PNS - M = 4.04, SD = .99
Creativity - M = 2.36, SD = .23
Sense of reassurance - M = 3.73, SD = 2.09

T-Test on the sense of reassurance between experiment and control is sig at .045. Standard Error = .378 [CI .017, 1.51].

For the moderation analysis, the PROCESS tool is being used (centering applied, bootstrapping etc).

Results for the relationship between PNS and Creativity when group is used as the moderator is p = .9254
Result for the relationship between PNS and Creativity when sense of reassurance is used as the moderator is p = .024, SE = ..0108, coeff = .0247, CI = .0033, .0461

Follow up simple slopes on second moderation is:
Reassurance = -2.057, effect = -.0525, SE = .0246, p = .0346 CI = .1012, -.0039
Reassurance = 0, effect = -0.0016, SE = .0236, p = .9448, CI = -.0485, .0452
Reassurance = 2.057, effect = .0493, SE = .0388, p = .2068, CI = -.0276, .1261

I hope that helps you understand what my data is looking like and where I might be going wrong?
 
#4
Results for the relationship between PNS and Creativity when group is used as the moderator is p = .9254
Result for the relationship between PNS and Creativity when sense of reassurance is used as the moderator is p = .024, SE = ..0108, coeff = .0247, CI = .0033, .0461
Initially you wrote that the respective moderator was statistically not significant in the first analysis, and statistically significant in the second analysis.
But here you report the first order effects of PNS->Creativity instead of the results for the moderators. I am not sure what to do with this.

With kind regards

Karabiner