Using regression to find "balance" (Making a video game. :))

#1
Hello everyone.
I'll try to be brief. The first paragraph or so may be irrelevant to answer the question, but I'm including it for context.

For school, a friend and I are developing a strategy game wherein combat units are constructed out of components that offer different benefits (movement speed, damage etc.). We're going to be using automated playtesting to "balance" these components, and I want to make sure I use a logical methodology to do so. From our playtest, we will be pitting a set of 'balanced' units against test-case units that use the new ability we are trying to balance. The components being tested will each have have 1-4 variable values for which we would want to find a balanced curve formula.

For my example, let's say we playtest a component with two variables, +Strength and +Dexterity. We playtest by randomly (within constraints) assigning bonus values to these components for hundreds of battles, throwing them into battle against unit(s) we eventually want to give a 50-50 chance of beating. Though there are random factors that can influence battle, obviously the battles where the bonus values are high give the unit a higher chance of winning and vice versa.

In a slight simplification of our tests, the results data would come out something like this, where STR and DEX stats are random and we get the W/L result of the playtest:

STR DEX W/L
14.3 04.9 W
03.7 07.3 L
01.0 17.8 L
10.2 09.6 W
19.6 06.3 W
02.9 16.1 W
...
And so on. From there we could plot W/L points on a scatter graph, and would want to create a logarithmic curve to define a roughly "balanced" ability, that plots X and Y coordinates for values of STR and DEX where the values on the curve give a 50/50 chance of winning (so p=0.5 in the regression). And beyond that, a methodology of doing so for components with more than 2 variables.

I hope I was clear, I'll reply to any clarification questions :)

I appreciate any help you could offer to help me in my research of the best/easiest way to calculate this. I've found a lot so far, but better to ask than to remain uncertain I'm going about it correctly.

Thank you!
 
#3
You may want to consider using a response surface approach instead of straight regression.
Thanks for your reply! My thoughts: Maybe? I just read through that thesis and if I understand it (a big if :)) I can't see if response surface is practical for what I'm seeking. They've got those continuous response variables for data and my data is just a boolean win or loss. Ha, as you might guess I've done regression for classes before but am worried about getting out of my depth. I apologize if I'm naive about the complexity of the goals of my anaylsis, I thought the best solution would be similar to the relatively straight regression stuff I've done before. I'll give surface response a further read; maybe I'll 'get it'.

However, If you (or anyone else) has any further/other help for me, please fire away!

Thanks
 

BGM

TS Contributor
#4
For a very elementary approach you may first look at logistic regression which you model the beating probability by several independent variable. Then you may want to find out the set of points which gives a probability of 0.5.

I am not asking for the secrets of your game but certainly the logit/linear relationship may not hold, and I suspect that the probability can be calculated in closed form - if your game is not very complex. You may try to guess the other models for regression.
 
#5
Thank you for that. Yes, if the components 'work' like I expect/hope, logistic regression is what I expect would best fit the results.

What's blocking me, (which I probably didn't explain clearly enough) is that I haven't yet figured out how to actually do the regression on my particular data set that I described at first. If it were a graph where I were trying to find a line that approximates the curve created by a set of data points that, that's textbook and I would have figured it out. But my data points are intentionally randomly distributed all over the place, and I'm trying to 'fit' my regression between two 'types' of data points, win or loss. And novice that I am, I haven't yet figured out how to do that.

I hope this forum is an appropriate place to ask - my request would be for a algorithmic description, or online reference reference that would help me figure this out. Again, thank you for your help so far and my apologies if I'm unclear or making a stupid request or whatever. :)
 
#6
Thank you for that. Yes, if the components 'work' like I expect/hope, logistic regression is what I expect would best fit the results.

What's blocking me, (which I probably didn't explain clearly enough) is that I haven't yet figured out how to actually do the regression on my particular data set that I described at first. If it were a graph where I were trying to find a line that approximates the curve created by a set of data points that, that's textbook and I would have figured it out. But my data points are intentionally randomly distributed all over the place, and I'm trying to 'fit' my regression between two 'types' of data points, win or loss. And novice that I am, I haven't yet figured out how to do that.

I hope this forum is an appropriate place to ask - my request would be for a algorithmic description, or online reference reference that would help me figure this out. Again, thank you for your help so far and my apologies if I'm unclear or making a stupid request or whatever. :)
 

BGM

TS Contributor
#7
Since you said you are the game developers, so I just suspect you will know the details for determining win/loss in the combat which may help to develop an exact formula, rather than estimating by a regression.

Also if I have read you table of result data correctly, your data type should be perfect for a logistic regression, as your data has a Binary response. There is no closed form solution for the estimators; all the coefficients has to be calculated numerically which is usually done by software.
 
#8
Hi BGM. Just to clarify:

Yes, we will be running the automated playtests which will determine the result of a given combat and give us out points on the graph.
We will need to do a regression of some sort because there are random elements built into battle. The results should/will reflect a trend/probability, not an absolute certainty of win/loss.
As far as calculating using software, since we're trying to design this balance 'machine', we're going to need to code the algorithm ourselves, and therefore need to know how to do it. :)

If you (or anyone else) knows how to do the P=50% regression for this sort of data set, could you describe the process, or find a reference for me? Thank you.
 

Dason

Ambassador to the humans
#10
I don't really think your pictures illustrate 'textbook logistic regression'... logistic regression uses a binary response and I don't see how that is pictured in your case. If in your second case the green/red are the actual responses and the blue line is the estimated 50% chance either way line then using an appropriate model that could be logistic regression.

I agree with BGM though in that if you actually know how the probability of success gets calculated then it might be possible to solve for this analytically. You keep saying that there is randomness and yes we understand that but there are ways of dealing with that.
 
#11
Cool, Dason, I think you're right - maybe I should have labeled it 'curve fitting', and maybe shouldn't have included that graph at all except to say "hey, I know how to do this, but not this."

Given the sort of scatter data points pictured in "what I want to do", I'm looking for an solid algorithm/ reference material that would help me calculate that blue curve.