mathematical model to build a ranking/ scoring system

vij

New Member
#1
I want to rank a set of sellers. Each seller is defined by parameters var1,var2,var3,var4...var20. I want to score each of the sellers.

Currently I am calculating score by assigning weights on these parameters(Say 10% to var1, 20 % to var2 and so on), and these weights are determined based on my gut feeling.

my score equation looks like

score = w1* var1 +w2* var2+...+w20*var20
score = 0.1*var1+ 0.5 *var2 + .05*var3+........+0.0001*var20
My score equation could also look like

score = w1^2* var1 +w2* var2+...+w20^5*var20
where var1,var2,..var20 are normalized.

Which equation should I use? What are the methods to scientifically determine, what weights to assign?

I want to optimize these weights to revamp the scoring mechanism using some data oriented approach to achieve a more relevant score.

example
I have following features for 1000 sellers

1] Order fulfillment rates [numeric]

2] Order cancel rate [numeric]

3] User rating [1-5] { 1-2 : Worst, 3: Average , 5: Good} [categorical]

4] Time taken to deliver order to customer. (shorter the time taken better is the seller) [numeric]

5] Price competitiveness

Are there better algorithms/approaches to solve this problem? calculating score? i.e I linearly added the various features, I want to know better approach to build the ranking system?

How to come with the values for the weights?

Apart from using above features, few more that I can think of are ratio of positive to negative reviews, rate of damaged goods etc. How will these fit into my Score equation?
 
#2
By now I expect you have discovered that the statistical approach to scoring is to have a performance measure such as net profit for each seller to function as the 'score' or dependent value to be modeled using your list of independent performance variables. Something has to function as the goal for which all are working. The rest of the things are measures of the things they do that contribute to the performance level they achieve. You would then have (total gross margin ~ order fulfillment rates + order cancel rate + user rating (satisfaction score?)). I suspect that it would be found that time taken to deliver orders to customers is correlated with satisfaction. Price competitiveness may be a different label for total gross margin or one is a multiple of the other. The modeling software will come up with the weighting values for each of the variables.