ncdl.nn.Mish

class ncdl.nn.Mish(*args: Any, **kwargs: Any)

Applies the Mish function, element-wise. Mish: A Self Regularized Non-Monotonic Neural Activation Function.

\[\text{Mish}(x) = x * \text{Tanh}(\text{Softplus}(x))\]

Note

See Mish: A Self Regularized Non-Monotonic Neural Activation Function

Shape:

  • Input: \((*)\), where \(*\) means any number of dimensions.

  • Output: \((*)\), same shape as the input.

scripts/activation_images/Mish.png

Examples:

>>> m = nn.Mish()
>>> input = torch.randn(2)
>>> output = m(input)
__init__(inplace: bool = False)

Methods

__init__([inplace])

extra_repr()

forward(input)

Attributes

inplace