what is the default kernel_initializer in keras
In the user manual, it shows the different kernel_initializer below
https://keras.io/initializers/
the main purpose is to initialize the weight matrix in the neural network.
Anyone knows what the default initializer is? the document didn't show the default.
Usually, it's glorot_uniform
by default. Different layer types might have different default kernel_initializer
. When in doubt, just look in the source code. For example, for Dense
layer:
class Dense(Layer):
...
def __init__(self, units,
activation=None,
use_bias=True,
kernel_initializer='glorot_uniform',
bias_initializer='zeros',
kernel_regularizer=None,
bias_regularizer=None,
activity_regularizer=None,
kernel_constraint=None,
bias_constraint=None,
**kwargs):
GlorotUniform, keras uses Glorot initialization with a uniform distribution.r = √(3/fan_avg)
fan_avg = (fan_in + fan_out) /2
number of inputs = fan_in
number of nurons in a layer = fan_out