% goals when your N is small

#1
This might be a dumb question but I figured I'd ask for some help/info. I work at a mental health agency and we have Outreach goals for clients who have severe mental health dxs. The problem is these clients are very hard to engage and the amount of referrals we receive is very small.

For example, we have a goal of 75% of our referrals will be seen in 14 days. We had 5 referrals in May and only saw 3 of them in 14 days (60%). Our average is 4 referrals a month. When we look at our data month to month, it seems almost impossible to reach our % goals when our N is so small.

My question is, when your N is so small, does it make sense to have a % goal? I was wondering if there was any literature/research on this topic.

Thanks for taking the time for reading this.
 

Dason

Ambassador to the humans
#2
I think this is more a logistics issue than a statistical one. From a hypothesis testing point of view 3 out of 5 isn't different enough from .75 to be statistically significant.

But I don't think that really gets at the issue. It sounds like higher ups want to have some sort of metric and they want to assess that at a set interval. It might make more sense to test once "N" patients come through the system as opposed to "N" weeks passing if you want to have at least some sort of confidence that the estimated percentage is "close" to the true percentage.
 

hlsmith

Less is more. Stay pure. Stay poor.
#3
Yeah, I had a project of decreasing c-diff infections once, but it is a rare event, so statistically hard to show change. In your case, what happens if you collapse these data into quarters (3-month groupings)? I would also, plot the number of days in a histogram to understand what is going on. You will want to make sure the axes are the same across periods so the bar heights aren't misleading.