Individual Covariate in Fractional Factorial Design

[titonio]

I want to make a marketing experiment (sending an e-mail) with fractional factorial design. I will send this e-mail to a mailing list consist of 660 members. I have 6 factors and use the 1/2 fraction. I plan to use response variable open rate (ratio between email open/sent) and CTR (ratio between click/open). But I have individual covariate data which is categorized in three categories (for example: director, manager, staff). If i use the response variable the data should be aggregate, and i should divide 660/32 = 20 recipient per e-mail. But how can I overcome with the covariate? Should I divide the covariate evenly to the groups? Or should I just use individual level data and change my response variable to click (click or no) and open (open or no). I am very confused about this.
Thank you!

 

[GretaGarbo]

I take it to be a two level factorial (since "32"). If there are 4 levels with "director, manager, staff, other" then they can be coded in the following way, by creating two pseudo variables:

category x1 x2
director, +1 +1
manager, -1 +1
staff, +1 -1
other -1 -1

Instead of 6 factors you will now have 6+2 =8 factors. With a two level factorial that gives 2^8 = 256 but you have room to run the full factorial and even to replicate it. So you can run a full replicated trial + an extra fraction like 512 + 128 = 640 runs.

I plan to use response variable open rate (ratio between email open/sent) and CTR (ratio between click/open).

So the response variable will be the the proportion so you can simply run it a a logit model. The dependent variabel will be coded as 0 (not open) or 1 (open).

 

[titonio]

I take it to be a two level factorial (since "32"). If there are 4 levels with "director, manager, staff, other" then they can be coded in the following way, by creating two pseudo variables:

category x1 x2
director, +1 +1
manager, -1 +1
staff, +1 -1
other -1 -1

Instead of 6 factors you will now have 6+2 =8 factors. With a two level factorial that gives 2^8 = 256 but you have room to run the full factorial and even to replicate it. So you can run a full replicated trial + an extra fraction like 512 + 128 = 640 runs.



So the response variable will be the the proportion so you can simply run it a a logit model. The dependent variabel will be coded as 0 (not open) or 1 (open).

Thank you for your reply! This really helps.

I want to ask again about the extra fraction, so if I want to add an extra fraction it should be 2^x? If there's some responses that I should exclude because it didn't pass a manipulation check, is it okay to run a logit model? (For example there are 622 data that's valid)

 

[GretaGarbo]

I want to ask again about the extra fraction, so if I want to add an extra fraction it should be 2^x?

Well, if you have access to say 48 experimental units and you have 5 factors, then 2^5 is 32. Then you can do an extra half fraction of 2^(5-1) =2^4 = 16. So then you have 32 + 16 =48. But if you have say 50 units then you can just randomize the two extra units to two experimental runs. It would not be the perfect orthogonal set up but they would give extra information.

About your 4 categories, maybe you have fewer directors than staff so that they don't fit in in the design, then you can just randomize them to some experimental runs.

 

[titonio]

Well, if you have access to say 48 experimental units and you have 5 factors, then 2^5 is 32. Then you can do an extra half fraction of 2^(5-1) =2^4 = 16. So then you have 32 + 16 =48. But if you have say 50 units then you can just randomize the two extra units to two experimental runs. It would not be the perfect orthogonal set up but they would give extra information.

About your 4 categories, maybe you have fewer directors than staff so that they don't fit in in the design, then you can just randomize them to some experimental runs.

Thank you again for your answer! One more question, is there any implication if I don't have the perfect orthogonal set up? Especially when I run logit model

 

[GretaGarbo]

No, compare this randomized experimental design with a register study (not designed and not randomized), where one compares exposed (for a chemical), smokers and lack of exercise. These variables will be correlated (and not orthogonal). But it is possible to evaluate them anyway. A randomized and balanced experimental design will be much better.

If you use a linear regression model the arrays of a fractional factorial (X'X) will be ortogonal. If you use a logit model they will be almost ortogonal because of (X'WX) but it will still be very good.

But still, you don't know if the sample size is large enough. If you want to detect a very small and tiny effect then maybe this sample size is not large enough.