Graziano and Raulin (8th ed)Graziano & Raulin
Research Methods (8th edition)

Reliability

Types of Reliability

There are three types of reliability:

  • test-retest reliability
  • interrater reliability
  • internal consistency reliability (coefficient alpha)

Computing Reliability

Both test-retest and interrater reliability are indexed with a product-moment correlation. The data file should have the two scores on the same line for each subject as shown in this screen. This file shows the scores for 12 subjects taken two weeks apart to estimate test-retest reliability, which is a measure of stability. 

If instead, the two scores represented separate ratings of two researcher assistants, we would use these data to compute interrater reliability. The statistical procedures are identical. 

To compute either test-retest or interrater reliability, select the Analyze menu, the Correlate submenu, and the Bivariate option to get this screen. This is the same screen for any bivariate correlation. Move the two scores that are part of the reliability analysis to the Variables box and click on OK to get the following screen

The correlation, in this case .949, is your reliability index. Reliability indices are traditionally rounded to two decimal places, so the reliability would be reported as .95 in this example.

If we assume that the time1 and time2 scores are to be summed to produce a total score, we essentially have a two-item test, and we can compute a measure of internal consistency. We can compute coefficient alpha, the index of internal consistency reliability, by selecting the Analyze menu, the Scale submenu, and the Reliability Analysis option, which will give us this screen. Moving the variables of time1 and time2 to the items box, making sure that "alpha" is selected as the model at the bottom of the box, and clicking OK will produce this output. In this case, the coefficient alpha reliability is .97.

Animations

We have prepared a series of animations that will walk you through the procedures discussed on this page. To run an animation, simply click on the title of the animation in the table below. 

Note that we do not recommend that you try to run the animations if you have a slow connection, such as a dial-up connection. You will find that the animations take forever to load with a slow connection.

Opening the Data File
Test-Retest or Interrater Reliability Analyses
Internal Consistency Reliability