How to Calculate Fleiss' Kappa

Written by peter flom
  • Share
  • Tweet
  • Share
  • Email

Fleiss' kappa is a statistical method of evaluating agreement among three or more raters rating any number of items. For example, suppose Abigail, Bob and Charlie each rate three paintings, by Picasso, Rembrandt and Monet, on a scale from 1 to 5. Fleiss' kappa corrects for the amount of chance variation.

Skill level:


  1. 1

    Count, for each item, the number of times each rating was given. For example, how many times was the Rembrandt painting given a rating of 5? If r is the number or raters and q is the number of rating categories, you should have q*r computations.

  2. 2

    Square these numbers.

  3. 3

    Add up the squared numbers.

  4. 4

    Subtract qr from the result. In the example, qr is 15. This result will be the numerator in a fraction

  5. 5

    Multiply rq by the number of raters - 1. In the example 152 = 30. This is the denominator of the fraction.

  6. 6

    Divide the numerator by the denominator. This is Fleiss' kappa.

Tips and warnings

  • You can get Fleiss kappa automatically in statistical programs such as R, SAS or SPSS.

Don't Miss

  • All types
  • Articles
  • Slideshows
  • Videos
  • Most relevant
  • Most popular
  • Most recent

No articles available

No slideshows available

No videos available
