medkit.text.metrics.irr_utils
=============================

.. py:module:: medkit.text.metrics.irr_utils

.. autoapi-nested-parse::

   Metrics to assess inter-annotator agreement.

   ..
       !! processed by numpydoc !!


Functions
---------

.. autoapisummary::

   medkit.text.metrics.irr_utils.krippendorff_alpha


Module Contents
---------------

.. py:function:: krippendorff_alpha(all_annotators_data: list[list[None | str | int]]) -> float

   
   Compute Krippendorff's alpha: a coefficient of agreement among many annotators.

   This coefficient is a generalization of several reliability indices.
   The general form is:

   .. math::
       \\alpha = 1 - \\frac{D_o}{D_e}

   where :math:`D_o` is the observed disagreement among labels assigned to
   units or annotations and :math:`D_e` is the disagreement between annotators
   attributable to chance. The arguments of the disagreement measures are values
   in coincidence matrices.

   This function implements the general computational form proposed in [Rc83e04bb9dc7-1]_,
   but only supports binaire or nominal labels.

   :Parameters:

       **all_annotators_data** : list of list of str or int or None
           Reliability_data, list or array of labels given to `n_samples` by `m_annotators`.
           Missing labels are represented with `None`



   :Returns:

       float
           The alpha coefficient, a number between 0 and 1.
           A value of 0 indicates the absence of reliability, and
           a value of 1 indicates perfect reliability.




   :Raises:

       AssertionError
           Raise if any list of labels within `all_annotators_data` differs in size or
           if there is only one label to be compared.





   .. rubric:: References

   .. [Rc83e04bb9dc7-1] K. Krippendorff, “Computing Krippendorff's alpha-reliability,”
           ScholarlyCommons, 25-Jan-2011, pp. 8-10. [Online].
           Available: https://repository.upenn.edu/asc_papers/43/

   .. only:: latex

      [Rc83e04bb9dc7-1]_

   .. rubric:: Examples

   Three annotators labelled six items. Some labels are missing.

   >>> annotator_A = ["yes", "yes", "no", "no", "yes", None]
   >>> annotator_B = [None, "yes", "no", "yes", "yes", "no"]
   >>> annotator_C = ["yes", "no", "no", "yes", "yes", None]
   >>> krippendorff_alpha([annotator_A, annotator_B, annotator_C])
   0.42222222222222217

   ..
       !! processed by numpydoc !!