model
Bases: model
The RPN model build in the tinyBIG toolkit.
It implements the RPN model build with a deep architecture involving multiple RPN layers.
Notes
The RPN model can be built with a deep architecture by stacking multiple RPN layers on top of each other. A deep RPN model with K layers can be represented as follows: $$ \begin{equation} \begin{cases} \text{Input: } & \mathbf{h}_0 = \mathbf{x},\\ \text{Layer 1: } & \mathbf{h}_1 = \left\langle \kappa_1(\mathbf{h}_0), \psi_1(\mathbf{w}_1) \right\rangle + \pi_1(\mathbf{h}_0),\\ \text{Layer 2: } & \mathbf{h}_2 = \left\langle \kappa_2(\mathbf{h}_1), \psi_2(\mathbf{w}_2) \right\rangle + \pi_2(\mathbf{h}_1),\\ \cdots & \cdots \ \cdots\\ \text{Layer K: } & \mathbf{h}_K = \left\langle \kappa_K(\mathbf{h}_{K-1}), \psi_K(\mathbf{w}_K) \right\rangle + \pi_K(\mathbf{h}_{K-1}),\\ \text{Output: } & \hat{\mathbf{y}} = \mathbf{h}_K. \end{cases} \end{equation} $$ Each of the layers shown above can be implemented with one RPN layer, which may also involve multi-heads and multi-channels.
Attributes:
Name | Type | Description |
---|---|---|
name |
str, default = 'Reconciled_Polynomial_Network'
|
Name of the RPN model. |
layers |
list, default = None
|
The model architecture with multiple layers. |
device |
str, default = 'cpu'
|
Device to host the RPN model. |
Methods:
Name | Description |
---|---|
__init__ |
The RPN model initialization method, it will initialize the model architecture based on the input configurations. |
get_depthber |
The RPN model depth retrieval method, it will get the number of layers involved in the RPN model. |
initialize_parameters |
The RPN model parameter initialization method, it will initialize the parameters of each layer. |
forward |
The forward method of the RPN model. It will generate the outputs based on the input data. |
Source code in tinybig/model/rpn.py
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 |
|
__init__(name='Reconciled_Polynomial_Network', layers=None, depth=None, depth_alloc=None, layer_configs=None, device='cpu', *args, **kwargs)
The initialization method of the RPN model with multiple RPN layers.
It initializes the deep RPN model composed with multiple RPN layers. Specifically, this method initializes the name, layers and devices to host the RPN model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
name
|
Name of the RPN model. |
'Reconciled_Polynomial_Network'
|
|
layers
|
list
|
The list of RPN layers in the model. The layers involved in the model can be initialized either directly with the layers parameter or via the layer_configs parameter. |
None
|
layer_configs
|
dict | list
|
The list of RPN layer detailed configurations. |
None
|
depth
|
int
|
The total layer number of the model. It is optional, if the "layers" or the "layer_configs" can provide sufficient information for the model initialization, this depth parameter can be set as None. |
None
|
depth_alloc
|
int | list
|
RPN allows the layers with different configurations, instead of listing such configurations one by one, it also allows the listing of each configuration types together with the repeating numbers for each of the layers, which are specified by this optional layer allocation parameter. |
None
|
device
|
str
|
The device for hosting the RPN layer. |
'cpu'
|
Returns:
Type | Description |
---|---|
object
|
The initialized RPN model object. |
Source code in tinybig/model/rpn.py
forward(x, device='cpu', *args, **kwargs)
The forward method of the RPN model.
It computes the desired outputs based on the data inputs. For the RPN with deep architecture involving multiple layers, this method will iteratively call each of the layers to process the data inputs, illustrated as follows:
\[ \begin{equation} \begin{cases} \text{Input: } & \mathbf{h}_0 = \mathbf{x},\\\\ \text{Layer 1: } & \mathbf{h}_1 = \left\langle \kappa_1(\mathbf{h}_0), \psi_1(\mathbf{w}_1) \right\rangle + \pi_1(\mathbf{h}_0),\\\\ \text{Layer 2: } & \mathbf{h}_2 = \left\langle \kappa_2(\mathbf{h}_1), \psi_2(\mathbf{w}_2) \right\rangle + \pi_2(\mathbf{h}_1),\\\\ \cdots & \cdots \ \cdots\\\\ \text{Layer K: } & \mathbf{h}_K = \left\langle \kappa_K(\mathbf{h}_{K-1}), \psi_K(\mathbf{w}_K) \right\rangle + \pi_K(\mathbf{h}_{K-1}),\\\\ \text{Output: } & \hat{\mathbf{y}} = \mathbf{h}_K. \end{cases} \end{equation} \]
Parameters:
Name | Type | Description | Default |
---|---|---|---|
x
|
Tensor
|
The input data instances. |
required |
device
|
Device for processing the data inputs. |
'cpu'
|
Returns:
Type | Description |
---|---|
Tensor
|
The desired outputs generated by the RPN model for the input data instances. |
Source code in tinybig/model/rpn.py
get_depth()
RPN model name retrieval method.
It returns the name of the RPN model.
Returns:
Type | Description |
---|---|
str
|
It returns the name of the RPN model. |
initialize_parameters(init_type='xavier_uniform', init_bias=True)
RPN model parameter initialization method.
It initializes the parameters of the RPN model with deep architectures. This method will call the "initialize_parameters" method for each of the involved layers.
Returns:
Type | Description |
---|---|
None
|
This method doesn't have any return values. |