Skip to content

metric

The base class of the evaluation metrics in the tinyBIG toolkit.

...

Attributes:

Name Type Description
name str, default = 'base_metric'

Name of the evaluation metric.

Methods:

Name Description
__init__

It performs the initialization of the evaluation metric.
evaluate

It performs the evaluation based on the inputs.
__call__

It reimplementation the build-in callable method.
Source code in tinybig/metric/base_metric.py 
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
class metric:
    """
    The base class of the evaluation metrics in the tinyBIG toolkit.

    ...

    Attributes
    ----------
    name: str, default = 'base_metric'
        Name of the evaluation metric.

    Methods
    ----------
    __init__
        It performs the initialization of the evaluation metric.

    evaluate
        It performs the evaluation based on the inputs.

    __call__
        It reimplementation the build-in callable method.
    """
    def __init__(self, name: str = 'base_metric', *args, **kwargs):
        """
        The initialization method of the base metric class.

        It initializes a metric object based on the provided method parameters.

        Parameters
        ----------
        name: str, default = 'base_metric'
            The name of the metric, with default value "base_metric".

        Returns
        ----------
        object
            The metric object.
        """
        self.name = name

    @staticmethod
    def from_config(configs: dict):
        if configs is None:
            raise ValueError("configs cannot be None")
        assert 'metric_class' in configs
        class_name = configs['metric_class']
        parameters = configs['metric_parameters'] if 'metric_parameters' in configs else {}
        return config.get_obj_from_str(class_name)(**parameters)

    def to_config(self):
        class_name = self.__class__.__name__
        attributes = {attr: getattr(self, attr) for attr in self.__dict__}

        return {
            "metric_class": class_name,
            "metric_parameters": attributes
        }

    @abstractmethod
    def evaluate(self, *args, **kwargs):
        """
        The evaluate method of the base metric class.

        It is declared to be an abstractmethod and needs to be implemented in the inherited evaluation metric classes.
        The evaluate method accepts prediction results and ground-truth results as inputs and return the evaluation metric scores as the outputs.

        Returns
        ----------
        float | dict
            The evaluation metric scores.
        """
        pass

    @abstractmethod
    def __call__(self, *args, **kwargs):
        """
        Reimplementation of the build-in callable method.

        It is declared to be an abstractmethod and needs to be implemented in the inherited evaluation metric classes.
        This callable method accepts prediction results and ground-truth results as inputs and return the evaluation metric scores as the outputs.

        Returns
        ----------
        float | dict
            The evaluation metric scores.
        """
        pass

__call__(*args, **kwargs) abstractmethod

Reimplementation of the build-in callable method.

It is declared to be an abstractmethod and needs to be implemented in the inherited evaluation metric classes. This callable method accepts prediction results and ground-truth results as inputs and return the evaluation metric scores as the outputs.

Returns:

Type Description
float | dict

The evaluation metric scores.
Source code in tinybig/metric/base_metric.py 
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
@abstractmethod
def __call__(self, *args, **kwargs):
    """
    Reimplementation of the build-in callable method.

    It is declared to be an abstractmethod and needs to be implemented in the inherited evaluation metric classes.
    This callable method accepts prediction results and ground-truth results as inputs and return the evaluation metric scores as the outputs.

    Returns
    ----------
    float | dict
        The evaluation metric scores.
    """
    pass

__init__(name='base_metric', *args, **kwargs)

The initialization method of the base metric class.

It initializes a metric object based on the provided method parameters.

Parameters:

Name Type Description Default
name str

The name of the metric, with default value "base_metric".

 'base_metric'

Returns:

Type Description
object

The metric object.
Source code in tinybig/metric/base_metric.py 
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
def __init__(self, name: str = 'base_metric', *args, **kwargs):
    """
    The initialization method of the base metric class.

    It initializes a metric object based on the provided method parameters.

    Parameters
    ----------
    name: str, default = 'base_metric'
        The name of the metric, with default value "base_metric".

    Returns
    ----------
    object
        The metric object.
    """
    self.name = name

evaluate(*args, **kwargs) abstractmethod

The evaluate method of the base metric class.

It is declared to be an abstractmethod and needs to be implemented in the inherited evaluation metric classes. The evaluate method accepts prediction results and ground-truth results as inputs and return the evaluation metric scores as the outputs.

Returns:

Type Description
float | dict

The evaluation metric scores.
Source code in tinybig/metric/base_metric.py 
73
74
75
76
77
78
79
80
81
82
83
84
85
86
@abstractmethod
def evaluate(self, *args, **kwargs):
    """
    The evaluate method of the base metric class.

    It is declared to be an abstractmethod and needs to be implemented in the inherited evaluation metric classes.
    The evaluate method accepts prediction results and ground-truth results as inputs and return the evaluation metric scores as the outputs.

    Returns
    ----------
    float | dict
        The evaluation metric scores.
    """
    pass