Kappa

Indicates the degree of agreement of the nominal or ordinal assessments made by multiple appraisers when evaluating the same samples. Kappa statistics are commonly used in cross tabulation (table) applications and in attribute agreement analysis.

For example, 45 patients are examined by two different doctors for a particular disease. How often will the doctor's diagnosis of the condition (positive or negative) agree? Another example of nominal assessments is inspectors rating defects on TV screens. Do they consistently agree on their classifications of bubbles, divets, and dirt?

Kappa values range from -1 to +1. The higher the value of kappa, the stronger the agreement.

When:

·    Kappa = 1, perfect agreement exists.

·    Kappa = 0, agreement is the same as would be expected by chance.

·    Kappa < 0, agreement is weaker than expected by chance; this rarely happens.

Typically a kappa value of at least 0.70 is required, but kappa values close to 0.90 are preferred.

When you have ordinal ratings, such as defect severity ratings on a scale of 1-5, Kendall's coefficients, which take ordering into consideration, are usually a more appropriate statistic to evaluate association than kappa alone.

Minitab can calculate both Fleiss's kappa and Cohen's kappa. Cohen's kappa assesses the degree of agreement when there are either two raters with a single trial or one rater with two trials. In Attribute Agreement Analysis, Minitab calculates Fleiss's kappa by default and offers the option to calculate Cohen's kappa when appropriate. In Cross Tabulation and Chi-Square, Minitab calculates only Cohen's kappa.

Fleiss's kappa and Cohen's kappa use different methods to estimate the probability that agreements occur by chance. For more information on the calculation of kappa statistics, see Methods and Formulas - Cross Tabulation and Chi-Square and Methods and Formulas - Attribute Agreement Analysis.