seqgra.evaluator.evaluator module

Abstract base class for all evaluators

class Evaluator(evaluator_id: str, evaluator_name: str, learner: seqgra.learner.learner.Learner, output_dir: str, supported_tasks: Optional[Set[str]] = None, supported_sequence_spaces: Optional[Set[str]] = None, supported_libraries: Optional[Set[str]] = None, silent: bool = False)[source]

Bases: abc.ABC

evaluate_model(set_name: str = 'test', subset_idx: Optional[List[int]] = None, subset_n: Optional[int] = None, subset_labels: Optional[List[str]] = None, subset_n_per_label: bool = True, subset_shuffle: bool = True, subset_threshold: Optional[float] = None, suppress_plots: bool = False)Any[source]
select_examples(set_name: str = 'test', labels: Optional[Set[str]] = None, threshold: Optional[float] = None)seqgra.schema.AnnotatedExampleSet[source]

Returns all correctly classified examples that exceed the threshold.

for the specified labels and set that exceed the threshold.

Parameters

TODO

Returns

TODO

select_n_examples(set_name: str = 'test', n: Optional[int] = None, labels: Optional[Set[str]] = None, n_per_label: bool = True, shuffle: bool = True, threshold: Optional[float] = None)seqgra.schema.AnnotatedExampleSet[source]
class FeatureImportanceEvaluator(evaluator_id: str, evaluator_name: str, learner: seqgra.learner.learner.Learner, output_dir: str, supported_tasks: Optional[Set[str]] = None, supported_sequence_spaces: Optional[Set[str]] = None, supported_libraries: Optional[Set[str]] = None, silent: bool = False)[source]

Bases: seqgra.evaluator.evaluator.Evaluator

evaluate_model(set_name: str = 'test', subset_idx: Optional[List[int]] = None, subset_n: Optional[int] = None, subset_labels: Optional[List[str]] = None, subset_n_per_label: bool = True, subset_shuffle: bool = True, subset_threshold: Optional[float] = None, suppress_plots: bool = False)Any[source]
select_examples(set_name: str = 'test', labels: Optional[Set[str]] = None, threshold: Optional[float] = None)seqgra.schema.AnnotatedExampleSet

Returns all correctly classified examples that exceed the threshold.

for the specified labels and set that exceed the threshold.

Parameters

TODO

Returns

TODO

select_n_examples(set_name: str = 'test', n: Optional[int] = None, labels: Optional[Set[str]] = None, n_per_label: bool = True, shuffle: bool = True, threshold: Optional[float] = None)seqgra.schema.AnnotatedExampleSet