reliability

  1. V

    Assessing quality of the questionnaire’s results

    Dear all, Consider the case that I issue a questionnaire to 100 subjects (e.g., some personality traits). I suspect that 10 of my subjects did not fill the questionnaire seriously, so I would like to exclude them. How can I statistically show that the quality of my results is better if I...
  2. L

    Agreement between raters / inter-rater reliability : Dilemma

    Hi, I have done a test where 2 different raters had to rate 1000 texts independently according to the following options: - Happy - Sad - Angry - Confused - Could not tell Now, I am trying to determine their agreement (i.e reliability). However, I am facing a dilemma -specially when the do...
  3. F

    Factor Analysis Question

    I am currently working on constructing a scale. The questions on the scale are measured on a scale from 0-100 in increments of 10, similar to the DES. I ran a factor analysis with my pool of items and eventually got to a scale I liked the psychometrics of. At this point, I realized I didn't...
  4. W

    Guidelines for building a statistical model for a business problem?

    Hi There, I would need some guidelines for creating a statistical, probabilistic model for a business problem I'm dealing with. Here is the problem: I have a website that connects tourists to tour guides. When a new inquiry comes, a tour guide is assigned (today quasi randomly) to the...
  5. N

    Measuring equivalence/reliability (ICC and consorts)

    For a reliability study I’m administering patient reported outcomes (questionnaires) with 2 modes of administration (on paper and via an electronic device). I’d like to measure equivalence/reliability of the 2 modes of administration. In other words: does the electronic version of the...
  6. P

    Assessing Reliability Between 2 Metrics

    I have constructed two alternative metrics for a particular phenomenon, and I am trying see whether they are approximately equal across the dataset (that is, whether they might be reliable "stand ins" for each other in a larger, later regression analysis, or if one is consistently different from...
  7. A

    How to calculate reliability of a change score (SAS)

    Hi all, I'm hoping you can lend me your expertise on this somewhat obscure problem. I have some repeated measures data (time 1 measurement, time 2 measurement, change/difference score). I'm interested in calculating the reliability of the change/difference score. Traditional scale...
  8. U

    Correction for attenuation: WAIS-IV and test X

    Hi, I have been asked to do a correction for attenuation of the context between a specific test (Lets call it test X) and a specific part-test of WAIS-IV (word understanding, lets call it test Y). The reliability for WAIS-IV word understanding (test Y) is r = 0,85 and we assume that the...
  9. A

    Quick Cronbach's Alpha Question

    Hi all, This may seem like a silly question, but I was wondering if I should remove outliers prior to running reliability analysis or run the tests with outliers included. I am hoping to get a Cronbach's alpha values for three questionnaires that I used in my research.
  10. M

    Dissertation Trouble Using Correlations

    Hello. I am having a problem with my dissertation and I was wondering if anyone is kind enough to help me out. I am running a within-subjects correlational study using two independent variables, Organizational Identity (OI) and Narcissistic Organizational Identity (NOI). I have 5 dependent...
  11. A

    reliability effects components of variance

    Hello TalkStats community. I'm currently stuck with a problem, concerning how reliability of a process effects the variance of the process. let me give you an example: Lets assume there is a machine that produces a product X. the machine and runs for 1000 hours. However, the machine has...
  12. O

    Evaluation of Bayesian Model

    How can I evaluate if a created bayesian model is reliable model that makes good predication? To conclude if the created bayesian model produces a low error rate.
  13. C

    Statistical Design of Experiments: Impact of missing and/or changing experiments

    Hello, I am new to both this forum and the concept of statistical design so please, treat me like the novice I am! OK, so I am hoping to carry out experimentation on a material. I am testing five different physical parameters of 4 different materials, all of the factors are numerical (i.e...
  14. S

    Comparing absolute values of difference scores between two groups

    I have two groups (A and B) that each completed a different version of an experimental measure (measure A and measure B). Both groups also completed a standard, "gold standard", measure. For the purposes of my study, I am treating the "gold standard" measure as a participant's true score. If I...
  15. R

    Split-Half Reliability Output: Where is the p value?

    Hi all, I've run a split-half reliability analysis on my data, but I'm not sure how to interpret the output. I know enough that the Spearman-Brown Coefficient for Equal Lengths will be my r, but what in the SPSS output should be reported as the p value? Sample output below: It's such a...
  16. B

    comparison of measurements of two methods - determination of reliability

    The aim is to compare two measurement methods. Therefore with each method two consecutive measurements were performed on 25 subjects (so overall 4 x 25 measured values) The relevant parameter is an amplitude. To determine the reliability of each method, the “interindividual coefficient of...
  17. D

    Reliability between sets - is the intraclass correlation coefficient appropriate?

    Hi, I am having trouble applying the ICC to data I have collected for my Masters thesis. I have collected 8 sets (of 30 samples) from 40+ participants. Data was collected with constant parameters for all participants in order to find the reliability of the instrument in question. (To clarify...
  18. P

    Reliability, validity, cronbach alpha figures for Method/measures section

    Hi, My research is on people on particular visas in the US, i.e., immigrants and mostly Asians. In my methods section when I write about the measures, where do I get the Reliability, validity, cronbach alpha figures from? Firstly, do I search for previous research on Asians or on...
  19. J

    Statistics of fault indicator and maintenance for repairable fleet

    There is a fleet of repairable units and the units have system health measurements which consist primarily of vibration amplitudes. The health monitoring system is "new", so the thresholds are based on standard deviations of "normal" operations. So, the thresholds are clearly too low...
  20. A

    RMSEA impacted by high reliability scores..

    Hello again Sorry - In relation to running Confirmatory Factor Analysis - I'm wondering whether someone is aware of RMSEA being impacted by high reliability scores ? As background, I understand that RMSEA can be affected by high factor loadings. Currently, I have a construct with 6...