Activations

Table of Contents

• Usage of Activations
• Available Activations

Usage of Activations

Activation function adds non-linearity to each layer's output. To specify the activation function of a layer. You could either appends after the layer an Activation layer

model = Sequential([Input(64), Dense(2), Activation('relu')])

or through the activation argument of the layer:

model = Sequential([Input(64), Dense(2, activation='relu')])

You can also pass a Theano/TensorFlow function to the actication argument:

from keraflow import backend as B

def truncated_relu(x):
    return B.relu(x, max_value=20)

model = Sequential([Input(64), Dense(2, activation=truncated_relu)])

or utilize an existing Keraflow activation function:

from keraflow import backend as B
from keraflow.activations import relu

def truncated_relu(x):
    return relu(x, max_value=20)

model = Sequential([Input(64), Dense(2, activation=truncated_relu)])

Available Activations

• linear
• sigmoid
• hard_sigmoid
• tanh
• relu
• softmax
• softplus
• softsign