Federated Learning: Privacy-Preserving Distributed Training

Module: Machine Learning | Difficulty: Advanced

Federated Averaging

Communication-Efficient

Differential Privacy

Secure Aggregation

import numpy as np

class FederatedAveraging:
    def __init__(self, global_model, clients):
        self.global_model = global_model
        self.clients = clients
    def train_round(self):
        local_weights = []
        for client in self.clients:
            client.set_weights(self.global_model.get_weights())
            client.train()
            local_weights.append(client.get_weights())
        # Weighted average
        avg_weights = []
        for i in range(len(local_weights[0])):
            weighted_sum = sum(w[i] * client.n_samples for w, client in zip(local_weights, self.clients))
            total_samples = sum(client.n_samples for client in self.clients)
            avg_weights.append(weighted_sum / total_samples)
        self.global_model.set_weights(avg_weights)
    def evaluate(self, test_data):
        return self.global_model.evaluate(test_data)

| Privacy | Accuracy | Communication | Scalability | |---------|----------|---------------|-------------| | None | High | Low | High | | DP | Medium | Medium | High | | Secure Agg | High | High | Medium | | HE | High | Very High | Low |

Research Insight: Federated learning can achieve 90%+ of centralized accuracy with 100 communication rounds. The key challenge is non-IID data, which causes client drift. FedProx and SCAFFOLD address this by adding proximal terms or variance reduction.