Objectives
Usage of objectives
An objective function (or loss function, or optimization score function) is one of the two parameters required to compile a model:
Scala:
model.compile(loss = "mean_squared_error", optimizer = "sgd")
Python:
model.compile(loss='mean_squared_error', optimizer='sgd')
Scala:
model.compile(loss = MeanSquaredError(sizeAverage = true), optimizer = "sgd")
Python:
model.compile(loss=MeanSquaredError(size_average=True), optimizer='sgd')
Available objectives
MeanSquaredError
The mean squared error criterion e.g. input: a, target: b, total elements: n
loss(a, b) = 1/n * sum(|a_i - b_i|^2)
Scala:
loss = MeanSquaredError(sizeAverage = true)
Parameters:
sizeAverage
a boolean indicating whether to divide the sum of squared error by n. Default: true
Python:
loss = MeanSquaredError(size_average=True)
Parameters:
size_average
a boolean indicating whether to divide the sum of squared error by n. Default: True
MeanAbsoluteError
Measures the mean absolute value of the element-wise difference between input and target
Scala:
loss = MeanAbsoluteError(sizeAverage = true)
Parameters:
sizeAverage
a boolean indicating whether to divide the sum of squared error by n. Default: true
Python:
loss = MeanAbsoluteError(size_average=True)
Parameters:
size_average
a boolean indicating whether to divide the sum of squared error by n. Default: True
BinaryCrossEntropy
Also known as logloss.
Scala:
loss = BinaryCrossEntropy(weights = null, sizeAverage = true)
Parameters:
weights
A tensor assigning weight to each of the classessizeAverage
whether to divide the sequence length. Default is true.
Python:
loss = BinaryCrossEntropy(weights=None, size_average=True)
Parameters:
weights
A tensor assigning weight to each of the classessize_average
whether to divide the sequence length. Default is True.
SparseCategoricalCrossEntropy
A loss often used in multi-class classification problems with SoftMax as the last layer of the neural network. By default, input(y_pred) is supposed to be probabilities of each class, and target(y_true) is supposed to be the class label starting from 0.
Scala:
loss = SparseCategoricalCrossEntropy(logProbAsInput = false, zeroBasedLabel = true, weights = null, sizeAverage = true, paddingValue = -1)
Parameters:
logProbAsInput
Boolean. Whether to accept log-probabilities or probabilities as input. Default is false and inputs should be probabilities.zeroBasedLabel
Boolean. Whether target labels start from 0. Default is true. If false, labels start from 1.weights
Tensor. Weights of each class if you have an unbalanced training set. Default is null.sizeAverage
Boolean. Whether losses are averaged over observations for each mini-batch. Default is true. If false, the losses are instead summed for each mini-batch.paddingValue
Integer. If the target is set to this value, the training process will skip this sample. In other words, the forward process will return zero output and the backward process will also return zero gradInput. Default is -1.
Python:
loss = SparseCategoricalCrossEntropy(log_prob_as_input=False, zero_based_label=True, weights=None, size_average=True, padding_value=-1)
Parameters:
log_prob_as_input
Boolean. Whether to accept log-probabilities or probabilities as input. Default is false and inputs should be probabilities.zero_based_label
Boolean. Whether target labels start from 0. Default is true. If false, labels start from 1.weights
A Numpy array. Weights of each class if you have an unbalanced training set. Default is None.size_average
Boolean. Whether losses are averaged over observations for each mini-batch. Default is True. If False, the losses are instead summed for each mini-batch.padding_value
Integer. If the target is set to this value, the training process will skip this sample. In other words, the forward process will return zero output and the backward process will also return zero gradInput. Default is -1.
MeanAbsolutePercentageError
Compute mean absolute percentage error for intput and target
Scala:
loss = MeanAbsolutePercentageError()
Parameters:
sizeAverage
a boolean indicating whether to divide the sum of squared error by n. Default: true
Python:
loss = MeanAbsolutePercentageError()
Parameters:
size_average
a boolean indicating whether to divide the sum of squared error by n. Default: True
MeanSquaredLogarithmicError
Compute mean squared logarithmic error for input and target
Scala:
loss = MeanSquaredLogarithmicError()
Python:
loss = MeanSquaredLogarithmicError()
CategoricalCrossEntropy
This is same with cross entropy criterion, except the target tensor is a one-hot tensor.
Scala:
loss = CategoricalCrossEntropy()
Python:
loss = CategoricalCrossEntropy()
Hinge
Creates a criterion that optimizes a two-class classification hinge loss (margin-based loss) between input x (a Tensor of dimension 1) and output y.
Scala:
loss = Hinge(margin = 1.0, sizeAverage = true)
Parameters:
margin
if unspecified, is by default 1.sizeAverage
whether to average the loss, is by default true
Python:
loss = Hinge(margin=1.0, size_average=True)
Parameters:
margin
if unspecified, is by default 1.size_average
whether to average the loss, is by default True
RankHinge
Hinge loss for pairwise ranking problems.
Scala:
loss = RankHinge(margin = 1.0)
Parameters:
margin
if unspecified, is by default 1.
Python:
loss = RankHinge(margin=1.0)
Parameters:
margin
if unspecified, is by default 1.
SquaredHinge
Creates a criterion that optimizes a two-class classification squared hinge loss (margin-based loss) between input x (a Tensor of dimension 1) and output y.
Scala:
loss = SquaredHinge(margin = 1.0, sizeAverage = true)
Parameters:
margin
if unspecified, is by default 1.sizeAverage
whether to average the loss, is by default true
Python:
loss = SquaredHinge(margin=1.0, size_average=True)
Parameters:
margin
if unspecified, is by default 1.size_average
whether to average the loss, is by default True
Poisson
Compute Poisson error for intput and target
Scala:
loss = Poisson()
Python:
loss = Poisson()
CosineProximity
Computes the negative of the mean cosine proximity between predictions and targets.
Scala:
loss = CosineProximity()
Python:
loss = CosineProximity()
KullbackLeiblerDivergence
Loss calculated as:
y_true = K.clip(y_true, K.epsilon(), 1)
y_pred = K.clip(y_pred, K.epsilon(), 1)
and output K.sum(y_true * K.log(y_true / y_pred), axis=-1)
Scala:
loss = KullbackLeiblerDivergence()
Python:
loss = KullbackLeiblerDivergence()