fibonacci_expansion

Bases: transformation

The fibonacci expansion function.

Applies Fibonacci polynomial expansion to input data.

Notes

Formally, the Fibonacci polynomials are a polynomial sequence that can be considered a generalization of the Fibonacci numbers, which can be recursively represented as follows:

Base cases \(n=0\) and \(n=1\):

\[
    \begin{equation}
    F_0(x) = 0 \text{, and } F_1(x) = 1.
    \end{equation}
\]

High-order cases with degree \(n \ge 2\):

\[
    \begin{equation}
    F_n(x) = x F_{n-1}(x) + F_{n-2}(x).
    \end{equation}
\]

Based on these recursive representations, we can illustrate some examples of the Fibonacci polynomials as follows:

\[
    \begin{equation}
    \begin{aligned}
    F_0(x) &= 0 \\
    F_1(x) &= 1 \\
    F_2(x) &= x \\
    F_3(x) &= x^2 + 1 \\
    F_4(x) &= x^3 + 2x \\
    F_5(x) &= x^4 + 3x^2 + 1 \\
    \end{aligned}
    \end{equation}
\]

Based on the above Fibonacci polynomials, we can define the data expansion functions as follows: $$ \begin{equation} \kappa(\mathbf{x} | d) = \left[ F_1(\mathbf{x}), F_2(\mathbf{x}), \cdots, F_d(\mathbf{x}) \right] \in R^D, \end{equation} $$ where the output dimension \(D = md\).

Attributes:

Name	Type	Description
d	int	The degree of Fibonacci polynomial expansion.

Methods:

Name	Description
calculate_D	Calculates the output dimension after expansion.
forward	Performs Fibonacci polynomial expansion on the input tensor.

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

class fibonacci_expansion(transformation):
    r"""
        The fibonacci expansion function.

        Applies Fibonacci polynomial expansion to input data.

        Notes
        ---------

        Formally, the Fibonacci polynomials are a polynomial sequence that can be considered a generalization of the Fibonacci numbers,
        which can be recursively represented as follows:

        __Base cases $n=0$ and $n=1$:__

        $$
            \begin{equation}
            F_0(x) = 0 \text{, and } F_1(x) = 1.
            \end{equation}
        $$

        __High-order cases with degree $n \ge 2$:__

        $$
            \begin{equation}
            F_n(x) = x F_{n-1}(x) + F_{n-2}(x).
            \end{equation}
        $$

        Based on these recursive representations, we can illustrate some examples of the Fibonacci polynomials as follows:

        $$
            \begin{equation}
            \begin{aligned}
            F_0(x) &= 0 \\
            F_1(x) &= 1 \\
            F_2(x) &= x \\
            F_3(x) &= x^2 + 1 \\
            F_4(x) &= x^3 + 2x \\
            F_5(x) &= x^4 + 3x^2 + 1 \\
            \end{aligned}
            \end{equation}
        $$

        Based on the above Fibonacci polynomials, we can define the data expansion functions as follows:
        $$
            \begin{equation}
            \kappa(\mathbf{x} | d) = \left[ F_1(\mathbf{x}), F_2(\mathbf{x}), \cdots, F_d(\mathbf{x}) \right] \in R^D,
            \end{equation}
        $$
        where the output dimension $D = md$.

        Attributes
        ----------
        d : int
            The degree of Fibonacci polynomial expansion.

        Methods
        -------
        calculate_D(m: int)
            Calculates the output dimension after expansion.
        forward(x: torch.Tensor, device='cpu', *args, **kwargs)
            Performs Fibonacci polynomial expansion on the input tensor.
    """
    def __init__(self, name='fibonacci_polynomial_expansion', d: int = 2, *args, **kwargs):
        """
            Initializes the Fibonacci polynomial expansion transformation.

            Parameters
            ----------
            name : str, optional
                Name of the transformation. Defaults to 'fibonacci_polynomial_expansion'.
            d : int, optional
                The maximum order of Fibonacci polynomials for expansion. Defaults to 2.
            *args : tuple
                Additional positional arguments.
            **kwargs : dict
                Additional keyword arguments.
        """
        super().__init__(name=name, *args, **kwargs)
        self.d = d

    def calculate_D(self, m: int):
        """
            Calculates the output dimension after Fibonacci polynomial expansion.

            Parameters
            ----------
            m : int
                Input dimension.

            Returns
            -------
            int
                Output dimension after expansion.
        """
        return m * self.d

    def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
        """
            Performs Fibonacci polynomial expansion on the input tensor.

            Parameters
            ----------
            x : torch.Tensor
                Input tensor of shape `(batch_size, input_dim)`.
            device : str, optional
                Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
            *args : tuple
                Additional positional arguments.
            **kwargs : dict
                Additional keyword arguments.

            Returns
            -------
            torch.Tensor
                Expanded tensor of shape `(batch_size, expanded_dim)`.

            Raises
            ------
            AssertionError
                If the output tensor shape does not match the expected dimensions.
        """

        b, m = x.shape
        x = self.pre_process(x=x, device=device)

        # base case: order 0
        expansion = torch.zeros(size=[x.size(0), x.size(1), self.d + 1]).to(device)
        # base case: order 1
        if self.d > 0:
            expansion[:, :, 1] = torch.ones(size=[x.size(0), x.size(1)]).to(device)
        # high-order cases
        for n in range(2, self.d + 1):
            expansion[:, :, n] = x * expansion[:, :, n-1].clone() + expansion[:, :, n-2].clone()

        expansion = expansion[:, :, 1:].contiguous().view(x.size(0), -1)

        assert expansion.shape == (b, self.calculate_D(m=m))
        return self.post_process(x=expansion, device=device)

__init__(name='fibonacci_polynomial_expansion', d=2, *args, **kwargs)

Initializes the Fibonacci polynomial expansion transformation.

Parameters:

Name	Type	Description	Default
name	str	Name of the transformation. Defaults to 'fibonacci_polynomial_expansion'.	'fibonacci_polynomial_expansion'
d	int	The maximum order of Fibonacci polynomials for expansion. Defaults to 2.	2
*args	tuple	Additional positional arguments.	()
**kwargs	dict	Additional keyword arguments.	{}

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

def __init__(self, name='fibonacci_polynomial_expansion', d: int = 2, *args, **kwargs):
    """
        Initializes the Fibonacci polynomial expansion transformation.

        Parameters
        ----------
        name : str, optional
            Name of the transformation. Defaults to 'fibonacci_polynomial_expansion'.
        d : int, optional
            The maximum order of Fibonacci polynomials for expansion. Defaults to 2.
        *args : tuple
            Additional positional arguments.
        **kwargs : dict
            Additional keyword arguments.
    """
    super().__init__(name=name, *args, **kwargs)
    self.d = d

calculate_D(m)

Calculates the output dimension after Fibonacci polynomial expansion.

Parameters:

Name	Type	Description	Default
m	int	Input dimension.	required

Returns:

Type	Description
int	Output dimension after expansion.

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

def calculate_D(self, m: int):
    """
        Calculates the output dimension after Fibonacci polynomial expansion.

        Parameters
        ----------
        m : int
            Input dimension.

        Returns
        -------
        int
            Output dimension after expansion.
    """
    return m * self.d

forward(x, device='cpu', *args, **kwargs)

Performs Fibonacci polynomial expansion on the input tensor.

Parameters:

Name	Type	Description	Default
x	Tensor	Input tensor of shape (batch_size, input_dim).	required
device	str	Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.	'cpu'
*args	tuple	Additional positional arguments.	()
**kwargs	dict	Additional keyword arguments.	{}

Returns:

Type	Description
Tensor	Expanded tensor of shape (batch_size, expanded_dim).

Raises:

Type	Description
AssertionError	If the output tensor shape does not match the expected dimensions.

Source code in tinybig/expansion/orthogonal_polynomial_expansion.py

def forward(self, x: torch.Tensor, device='cpu', *args, **kwargs):
    """
        Performs Fibonacci polynomial expansion on the input tensor.

        Parameters
        ----------
        x : torch.Tensor
            Input tensor of shape `(batch_size, input_dim)`.
        device : str, optional
            Device for computation ('cpu', 'cuda'). Defaults to 'cpu'.
        *args : tuple
            Additional positional arguments.
        **kwargs : dict
            Additional keyword arguments.

        Returns
        -------
        torch.Tensor
            Expanded tensor of shape `(batch_size, expanded_dim)`.

        Raises
        ------
        AssertionError
            If the output tensor shape does not match the expected dimensions.
    """

    b, m = x.shape
    x = self.pre_process(x=x, device=device)

    # base case: order 0
    expansion = torch.zeros(size=[x.size(0), x.size(1), self.d + 1]).to(device)
    # base case: order 1
    if self.d > 0:
        expansion[:, :, 1] = torch.ones(size=[x.size(0), x.size(1)]).to(device)
    # high-order cases
    for n in range(2, self.d + 1):
        expansion[:, :, n] = x * expansion[:, :, n-1].clone() + expansion[:, :, n-2].clone()

    expansion = expansion[:, :, 1:].contiguous().view(x.size(0), -1)

    assert expansion.shape == (b, self.calculate_D(m=m))
    return self.post_process(x=expansion, device=device)