Ctrl+K

medkit.text.ner.hf_entity_matcher

medkit.text.ner.hf_entity_matcher#

This module needs extra-dependencies not installed as core dependencies of medkit. To install them, use pip install medkit-lib[hf-entity-matcher].

Classes:

HFEntityMatcher(model[, ...])

Entity matcher based on HuggingFace transformers model
class HFEntityMatcher(model, aggregation_strategy='max', attrs_to_copy=None, device=-1, batch_size=1, hf_auth_token=None, cache_dir=None, name=None, uid=None)[source]#

Entity matcher based on HuggingFace transformers model

Any token classification model from the HuggingFace hub can be used (for instance “samrawal/bert-base-uncased_clinical-ner”).

Parameters:

  • model (str or Path) – Name (on the HuggingFace models hub) or path of the NER model. Must be a model compatible with the TokenClassification transformers class.

  • aggregation_strategy (str, default="max") – Strategy to fuse tokens based on the model prediction, passed to TokenClassificationPipeline. Defaults to “max”, cf https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.TokenClassificationPipeline.aggregation_strategy for details

  • attrs_to_copy (list of str, optional) – Labels of the attributes that should be copied from the input segment to the created entity. Useful for propagating context attributes (negation, antecendent, etc).

  • device (int, default=-1) – Device to use for the transformer model. Follows the HuggingFace convention (-1 for “cpu” and device number for gpu, for instance 0 for “cuda:0”).

  • batch_size (int, default=1) – Number of segments in batches processed by the transformer model.

  • hf_auth_token (str, optional) – HuggingFace Authentication token (to access private models on the hub)

  • cache_dir (str or Path, optional) – Directory where to store downloaded models. If not set, the default HuggingFace cache dir is used.

  • name (str, optional) – Name describing the matcher (defaults to the class name).

  • uid (str, optional) – Identifier of the matcher.

Methods:

make_trainable(model_name_or_path, labels, ...)

Return the trainable component of the operation.

run(segments)

Return entities for each match in segments.

set_prov_tracer(prov_tracer)

Enable provenance tracing.

Attributes:

description

Contains all the operation init parameters.
run(segments)[source]#

Return entities for each match in segments.

Parameters:

segments (list of Segment) – List of segments into which to look for matches.

Return type:

list[Entity]

Returns:

list of Entity – Entities found in segments.

static make_trainable(model_name_or_path, labels, tagging_scheme, tag_subtokens=False, tokenizer_max_length=None, hf_auth_token=None, device=-1)[source]#

Return the trainable component of the operation. This component can be trained using Trainer, and then used in a new HFEntityMatcher operation.

property description: OperationDescription#

Contains all the operation init parameters.

Return type:

OperationDescription

set_prov_tracer(prov_tracer)#

Enable provenance tracing.

Parameters:

prov_tracer (ProvTracer) – The provenance tracer used to trace the provenance.

Contents