ncdl.nn.LeakyReLU

class ncdl.nn.LeakyReLU(*args: Any, **kwargs: Any)

Applies the element-wise function:

\[\text{LeakyReLU}(x) = \max(0, x) + \text{negative\_slope} * \min(0, x)\]

or

\[\begin{split}\text{LeakyReLU}(x) = \begin{cases} x, & \text{ if } x \geq 0 \\ \text{negative\_slope} \times x, & \text{ otherwise } \end{cases}\end{split}\]
Parameters:

  • negative_slope – Controls the angle of the negative slope. Default: 1e-2

  • inplace – can optionally do the operation in-place. Default: False

Shape:

  • Input: \((*)\) where * means, any number of additional dimensions

  • Output: \((*)\), same shape as the input

scripts/activation_images/LeakyReLU.png

Examples:

>>> m = nn.LeakyReLU(0.1)
>>> input = torch.randn(2)
>>> output = m(input)
__init__(negative_slope: float = 0.01, inplace: bool = False) → None

Methods

__init__([negative_slope, inplace])

extra_repr()

forward(input)

Attributes

inplace

negative_slope