t test or ANOVA for my survey results?

#1
Hi all.
I ran a survey where I showed 2 ads to men and women in 5 different countries to see which they preferred. I want to know if gender or country influenced the selection of a particular ad preference. The data is normal.

I was told to use a One-way ANOVA but I don't think this is correct as I have 2 numerical categories (number of people who selected each concept) and the rest are categorical - basically, I don't have 3 columns of numbers, only 2.

Then, I explored using a paired t-test, but i'm not sure how to perform this to see differences between ads AND countries - will I have to run multiple tests between 2 countries to find one with a significant difference?

Any advice would be fab!
Thanks
 

Karabiner

TS Contributor
#2
The data is normal.
That makes not awfully much sense, I'm afraid, since you only have categorical variables
(gender, country, preference for either A or B). Normality (which by the way is not
relevant for nearly any statistical test) can only be observed in interval scaled variables.

How large is your sampe size, by the way?
I was told to use a One-way ANOVA but I don't think this is correct as I have 2 numerical categories (number of people who selected each concept) and the rest are categorical - basically, I don't have 3 columns of numbers, only 2.
You are correct that you cannot use a oneway analysis (which uses only 1 factor), because
you have 2 factors. So one would perfom a 2-factorial analysis of variance. But since
your dependent variable is categorical (selection of either A or B), analysis of variance cannot
be used. Analysis of variance requires an interval scaled dependent variable.
Then, I explored using a paired t-test, but i'm not sure how to perform this to see differences between ads AND countries - will I have to run multiple tests between 2 countries to find one with a significant difference?
Could you tell us a little bit about your background - how much you have learned about
statistics so far, why you are doing this study, and who will receive the results?

With knd regards

Karabiner
 
Last edited:
#3
That makes not awfully much sense, I'm afraid, since you only have categorical variables
(gender, country, preference for either A or B). Normality (which by the way is not
relevant for nearly any statistical test) can only be observed in interval scaled variables.

How large is your sampe size, by the way?

You are correct that you cannot use a oneway analysis (which uses only 1 factor), because
you have 2 factors. So one would perfom a 2-factorial analysis of variance. But since
your dependent variable is categorical (selection of either A or B), analysis of variance cannot
be used. Analysis of variance requires an interval scaled dependent variable.

Could you tell us a little bit about your background - how much you have learned about
statistics so far, why you are doing this study, and who will receive the results?

With knd regards

Karabiner
Thank you so much for your insightful reply! My sample size is 129, with 61 men and 68 women. I have numerical data in the form of frequency of response! I ran the normality test on a table that had the columns: Country, Ad 1, Ad 2, then the associated numbers if that makes sense. I got a normal dis according to Shapiro Wilk!

I used to be a scientist and worked with purely quant data, but I am now a junior analyst that works with qualitative data. This is for a client project i’m working on and my colleague insists on stats. I’ve done some informal courses online (Harvard R and statistics) and a few modules at university, but nothing recently which is why I’m a bit rusty…

Would you agree with the suggestion of using a paired t test multiple times? I was thinking of using this method and comparing 2 of the countries at a time to see if there is a difference, but this also means i’ll have very few numbers per comparison so I’m confused if it’s even valid, eg (6 people like Ad 1, 6 like Ad 2 in France and 7 like Ad 1 but 5 like Ad 2 in Italy compared with the T test).

I hope that makes sense.
 

Karabiner

TS Contributor
#4
Thank you so much for your insightful reply! My sample size is 129, with 61 men and 68 women. I have numerical data in the form of frequency of response! I ran the normality test on a table that had the columns: Country, Ad 1, Ad 2, then the associated numbers if that makes sense. I got a normal dis according to Shapiro Wilk!
No, that does not make any sense at all, unfortunately.

It would be useful if you had a datafile with 129 individual subjects, and for each subject, you have 3 informations
(gender, country, selection). For example, 129 rows and 3 columns.

You could then start your analysis with 2 crosstabulations:, gender x selection, and contry x selection. This will show
you the associations between the respective variables. If you need a statistical test, you can try Chi².

If you don't have the individual data, only the aggregatet tables, you could maybe find an online calculator
for cross-tables where you can enter the aggregated data.

Would you agree with the suggestion of using a paired t test multiple times?
No. The t-test needs an interval scaled dependent variable. Your dependent variable is binary (choice A versus B).

With kind regards

Karabiner
 
#5
No, that does not make any sense at all, unfortunately.

It would be useful if you had a datafile with 129 individual subjects, and for each subject, you have 3 informations
(gender, country, selection). For example, 129 rows and 3 columns.

You could then start your analysis with 2 crosstabulations:, gender x selection, and contry x selection. This will show
you the associations between the respective variables. If you need a statistical test, you can try Chi².

If you don't have the individual data, only the aggregatet tables, you could maybe find an online calculator
for cross-tables where you can enter the aggregated data.


No. The t-test needs an interval scaled dependent variable. Your dependent variable is binary (choice A versus B).

With kind regards

Karabiner
Ah, so are you suggesting I should put the data as is and not summarised into one number? So I would have IDs of each respondent and then their response in terms of those categories? And I would basically run Chi^2 2 times, for each grouping of variables?

This would work great for the gender as it would tell me if that influences the choice, but I have 5 countries and would ideally like to know which one is impacting which. Should I then do the Chi sq multiple times on the small tables, each time including 2 sets country data?

Once again, thank you so much!

Also if you have any recommendations on books/courses to explore, I would be super grateful!
 

Karabiner

TS Contributor
#6
With 2 gender categories x 5 countries x 2 response categories = 20 possible combinations, a sample size such as n=129
is sparse. On average, only 6 observations are expected in each cell. I suppose that this is too few to correctely carry out
an analysis which includes both predictors at the same time.

Since I am not active in statistics education, unfortunately I cannot recommend books or courses. In the past I frequently
read reports from beginners that Andy Field's "Discovering tatistics ..." was useful for them.

With kind regards

Karabiner
 
Last edited:
#7
You can do a logit analysis with the ad as a dependent variable and with gender and country as explanatory variables. Search the internet for explanations and how to create dummy variables for gender and country.

With 2 gender categories x 5 countries x 2 response categories = 20 possible combinations, a sample size such as n=129
is sparse. On average, only 6 observations are expected in each cell. I suppose that this is too few to correctely carry out
an analysis which includes both predictors at the same time.
I am more optimistic. Even with an interaction effect between gender and country I believe that a likelihood ratio test would still be approximately chi squared distributed so that a test can be done.