Performance Management! - HR and Statistics???

Morning all,

Hopefully a subject line that *may* attract some views and then interests.

I have a real life problem(dont we all!)

I am working on a Performance Management system for my department in a software company and I am up against a brick wall that seems to not be giving way.

Situation is this:

I rate my staff on certain criteria, these are general and specific, for example

General criteria can be stuff like communcation skills, absence etc. This is rated on a scale of 1 t 4 ( I just chose 1 to 4 as its easier to define than 1 to 5 where I get staff who sit on the fence and choose 3 as most of the answers.)

Each Job role then has a set of specific criteria as well. Each of these criteria has a weighting based on how important that particular task is.

Now in an ideal world, if they are able to fill up all the criteria then all will be well.

From a HR perspective - I essentially, get them to rate themselves, then their Manager, will in turn rate them, both employee and manager will discuss, negotiate, fight, scream shout and essentailly finally agree on a rating. This in turn will give a final score that is then used as a measure of their performance.

The Problem:

In specific criteria - say we have Criteria 1(C1) to C5 on which they should be rated on(bear in mind its weighted), there may be circumstances where a person A will not have done any work related to C3 and hence will only have a rating for C1, C2, C4 and C5. Now tro compare this person to someone else Person B who has all ratings will be unfair right?

How do we get around this? Is there a statistical method to use? I may even be posting this on the wrong forum completely! :-( sorry If I am, but am sure there is something in stats that can help me out, vaguely remember studying this in college, but cant seem to remember how and what I did! :-(

Methods / ways I can think of is
a) give them the average of that skill set / criteria based on other people in that same job role.
b) Ignore it and divide the weighting among the other criteria??? I dunno - just thinking loud to see if I can come up with something....

Just confused and want to see IF there is an answer to this somewhere at all....

Its not a homework question, i am not even sure if statistics can help, but hey one can learn only by asking for answers, so lets see if someone else has come across this or if there is actually a method to get around this.

I can provide as much detail as necessary as its all in my head, and I can make assumptions, or change the way we do it as its in design stage now.

Hope the 'truth is out there'