Loss Functions

Introduction

Loss functions measure the difference between predictions and targets, guiding model optimization.

Common Losses

# Classification losses
model.compile(optimizer='adam', loss='categorical_crossentropy')  # One-hot labels
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')  # Integer labels

# Binary classification
model.compile(optimizer='adam', loss='binary_crossentropy')

# Regression losses
model.compile(optimizer='adam', loss='mse')
model.compile(optimizer='adam', loss='mae')
model.compile(optimizer='adam', loss='mse')  # MSE

Custom Loss

import tensorflow as tf

def huber_loss(y_true, y_pred, delta=1.0):
    error = y_true - y_pred
    abs_error = tf.abs(error)
    quadratic = tf.minimum(abs_error, delta)
    linear = abs_error - quadratic
    return tf.reduce_mean(0.5 * quadratic**2 + delta * linear)

model.compile(optimizer='adam', loss=huber_loss)

Multiple Losses

# Multi-task learning
model = keras.Model(inputs, [output1, output2])
model.compile(
    optimizer='adam',
    loss={'output1': 'mse', 'output2': 'binary_crossentropy'},
    loss_weights={'output1': 1.0, 'output2': 0.5}
)

Custom Metric

def custom_metric(y_true, y_pred):
    return tf.reduce_mean(tf.abs(y_true - y_pred))

model.compile(optimizer='adam', loss='mse', metrics=[custom_metric])

Practice Problems

1. Use categorical crossentropy
2. Implement custom loss function
3. Add multiple loss functions
4. Create custom metric
5. Handle class imbalance with weights