EvaluationHandler

class mouffet.evaluation.evaluation_handler.EvaluationHandler(*args, **kwargs)

Base class for evaluating models. Inherits ModelHandler

Relevant options:

<Global> data_config: Path to the data configuration file used to initialize the data handler. evaluation_dir: Directory where to save results

<Models> model_dir: Directory where to load models. predictions_dir: Directory where to load/save predictions. reclassify: Run the model again even if a prediction file is found

<Evaluators> type: The type of evaluator corresponding to one of the keys of EVALUATORS attribute of the subclass

Parameters:

ModelHandler ([type]) – [description]

Raises:

AttributeError – [description]

Returns:

[description]

Return type:

[type]

abstract predict_database(model, database, db_type='test')

This function calls a model to get predictions for an entire database. The data to be classified is usually loaded there, since predictions can be saved to avoid the reclassification. This avoids loading the data for nothing. This function also logs general information about the classification that is stored in the infos dict

Parameters:

• model (_type_) – _description_

• database (_type_) – _description_

• db_type (str, optional) – _description_. Defaults to “test”.

Returns:

a tuple containing an instance of pandas Dataframe with the predictions given by the model and a dict with general informations about the classification process (e.g. number of items process, time spent during the classification, etc.)

Return type:

tuple