medkit.text.metrics.classification
==================================

.. py:module:: medkit.text.metrics.classification


Classes
-------

.. autoapisummary::

   medkit.text.metrics.classification.TextClassificationEvaluator


Module Contents
---------------

.. py:class:: TextClassificationEvaluator(attr_label: str)

   
   An evaluator for attributes of TextDocuments.
















   ..
       !! processed by numpydoc !!

   .. py:method:: _extract_attr_values(docs: list[medkit.core.text.TextDocument]) -> list[str | int | bool]

      
      Prepare docs attrs to compute the metric.


      :Parameters:

          **docs** : list of TextDocument
              List of documents with attributes

      :Returns:

          list of str or int or bool
              List with the representation of the attribute by document.













      ..
          !! processed by numpydoc !!


   .. py:method:: compute_classification_report(true_docs: list[medkit.core.text.TextDocument], predicted_docs: list[medkit.core.text.TextDocument], metrics_by_attr_value: bool = True, average: typing_extensions.Literal[macro, weighted] = 'macro') -> dict[str, float | int]

      
      Compute classification metrics of document attributes giving annotated documents.

      This method uses `sklearn.metrics.classification_report` to compute
      precision, recall and F1-score for value of the attribute.

      .. warning::
          The set of true and predicted documents must be sorted to calculate the metric

      :Parameters:

          **true_docs** : list of TextDocument
              Text documents containing attributes of reference

          **predicted_docs: list of TextDocument**
              Text documents containing predicted attributes

          **metrics_by_attr_value: bool, default=True**
              Whether return metrics by attribute value.
              If False, only global metrics are returned

          **average** : str, default="macro"
              Type of average to be performed in metrics.
              - `macro`, unweighted mean (default)
              - `weighted`, weighted average by support (number of true instances by attr value)

      :Returns:

          dict of str to float or int
              A dictionary with the computed metrics













      ..
          !! processed by numpydoc !!


   .. py:method:: compute_cohen_kappa(docs_annotator_1: list[medkit.core.text.TextDocument], docs_annotator_2: list[medkit.core.text.TextDocument]) -> dict[str, float | int]

      
      Compute the cohen's kappa score, an inter-rated agreement score between two annotators.

      This method uses 'sklearn' as backend to compute the level of agreement.

      .. warning::
          The set of documents must be sorted to calculate the metric

      :Parameters:

          **docs_annotator_1** : list of TextDocument
              Text documents containing attributes annotated by the first annotator

          **docs_annotator_2** : list of TextDocument
              Text documents to compare, these documents contain attributes
              annotated by the other annotator

      :Returns:

          dict of str to float or int
              A dictionary with cohen's kappa score and support (number of annotated docs).
              The value is a number between -1 and 1, where 1 indicates perfect agreement; zero
              or lower indicates chance agreement.













      ..
          !! processed by numpydoc !!


   .. py:method:: compute_krippendorff_alpha(docs_annotators: list[list[medkit.core.text.TextDocument]]) -> dict[str, float | int]

      
      Compute the Krippendorff alpha score, an inter-rated agreement score between multiple annotators.

      .. warning::
          Documents must be sorted to calculate the metric.

      .. note::
          See :mod:`medkit.text.metrics.irr_utils.krippendorff_alpha` for more information about the score

      :Parameters:

          **docs_annotators** : list of list of TextDocument
              A list of list of Text documents containing attributes.
              The size of the list is the number of annotators to compare.

      :Returns:

          dict of str to float or int
              A dictionary with the krippendorff alpha score, number of annotators and support (number of documents).
              A value of 1 indicates perfect reliability between annotators; zero or lower indicates
              absence of reliability.













      ..
          !! processed by numpydoc !!