System Design Fundamentals: Load Balancing Strategies

Load balancing is a critical component of system design that distributes incoming requests across multiple servers to ensure optimal resource utilization, minimize response time, and avoid overload of any single resource.

What is Load Balancing?

Load balancing refers to the process of distributing network or application traffic across multiple servers. A load balancer acts as a "traffic cop" sitting in front of your servers and routing client requests across all servers capable of fulfilling those requests.

## Types of Load Balancers

1. Layer 4 (Transport Layer) Load Balancing
Operates at the transport layer and makes routing decisions based on IP and port information.

// Example: Simple round-robin implementation
class Layer4LoadBalancer {
  constructor(servers) {
    this.servers = servers;
    this.currentIndex = 0;
  }

  getNextServer() {
    const server = this.servers[this.currentIndex];
    this.currentIndex = (this.currentIndex + 1) % this.servers.length;
    return server;
  }
}

const servers = ['192.168.1.10:8080', '192.168.1.11:8080', '192.168.1.12:8080'];
const loadBalancer = new Layer4LoadBalancer(servers);

### 2. Layer 7 (Application Layer) Load Balancing
Makes routing decisions based on application-level data such as HTTP headers, URLs, or cookies.

class Layer7LoadBalancer {
  constructor() {
    this.routes = new Map();
  }

  addRoute(path, servers) {
    this.routes.set(path, servers);
  }

  routeRequest(request) {
    const path = new URL(request.url).pathname;
    
    // Route based on path
    if (path.startsWith('/api/')) {
      return this.routes.get('/api/');
    } else if (path.startsWith('/static/')) {
      return this.routes.get('/static/');
    }
    
    return this.routes.get('default');
  }
}

## Load Balancing Algorithms

### 1. Round Robin
Distributes requests sequentially across servers.

class RoundRobinBalancer:
    def __init__(self, servers):
        self.servers = servers
        self.current = 0
    
    def get_server(self):
        server = self.servers[self.current]
        self.current = (self.current + 1) % len(self.servers)
        return server

### 2. Weighted Round Robin
Assigns different weights to servers based on their capacity.

class WeightedRoundRobinBalancer:
    def __init__(self, servers_weights):
        self.servers_weights = servers_weights  # [(server, weight), ...]
        self.current_weights = [0] * len(servers_weights)
    
    def get_server(self):
        total_weight = sum(weight for _, weight in self.servers_weights)
        
        for i, (server, weight) in enumerate(self.servers_weights):
            self.current_weights[i] += weight
            
            if self.current_weights[i] >= max(self.current_weights):
                self.current_weights[i] -= total_weight
                return server

### 3. Least Connections
Routes to the server with the fewest active connections.

class LeastConnectionsBalancer:
    def __init__(self, servers):
        self.servers = servers
        self.connections = {server: 0 for server in servers}
    
    def get_server(self):
        return min(self.connections, key=self.connections.get)
    
    def add_connection(self, server):
        self.connections[server] += 1
    
    def remove_connection(self, server):
        self.connections[server] = max(0, self.connections[server] - 1)

## Health Checks

Implementing health checks ensures traffic is only sent to healthy servers.

class HealthChecker {
  constructor(servers, checkInterval = 30000) {
    this.servers = servers;
    this.healthyServers = new Set(servers);
    this.checkInterval = checkInterval;
    this.startHealthChecks();
  }

  async checkServerHealth(server) {
    try {
      const response = await fetch(http://${server}/health, {
        timeout: 5000
      });
      return response.ok;
    } catch (error) {
      return false;
    }
  }

  async startHealthChecks() {
    setInterval(async () => {
      for (const server of this.servers) {
        const isHealthy = await this.checkServerHealth(server);
        
        if (isHealthy) {
          this.healthyServers.add(server);
        } else {
          this.healthyServers.delete(server);
        }
      }
    }, this.checkInterval);
  }

  getHealthyServers() {
    return Array.from(this.healthyServers);
  }
}

## Session Persistence (Sticky Sessions)

For applications that maintain session state, you might need sticky sessions.

class StickySessionBalancer {
  constructor(servers) {
    this.servers = servers;
    this.sessionMap = new Map();
    this.roundRobin = new RoundRobinBalancer(servers);
  }

  getServer(sessionId) {
    if (this.sessionMap.has(sessionId)) {
      return this.sessionMap.get(sessionId);
    }

    const server = this.roundRobin.getServer();
    this.sessionMap.set(sessionId, server);
    return server;
  }

  removeSession(sessionId) {
    this.sessionMap.delete(sessionId);
  }
}

## Load Balancer Implementation Example

Here's a complete example using Node.js:

const http = require('http');
const httpProxy = require('http-proxy');

class LoadBalancer {
  constructor(servers) {
    this.servers = servers;
    this.current = 0;
    this.proxy = httpProxy.createProxyServer({});
    
    // Handle proxy errors
    this.proxy.on('error', (err, req, res) => {
      console.error('Proxy error:', err);
      res.writeHead(500, { 'Content-Type': 'text/plain' });
      res.end('Internal Server Error');
    });
  }

  getNextServer() {
    const server = this.servers[this.current];
    this.current = (this.current + 1) % this.servers.length;
    return server;
  }

  handleRequest(req, res) {
    const target = this.getNextServer();
    console.log(Routing request to: ${target});
    
    this.proxy.web(req, res, {
      target: http://${target}
    });
  }

  start(port = 3000) {
    const server = http.createServer((req, res) => {
      this.handleRequest(req, res);
    });

    server.listen(port, () => {
      console.log(Load balancer running on port ${port});
      console.log(Balancing between: ${this.servers.join(', ')});
    });
  }
}

// Usage
const servers = ['localhost:3001', 'localhost:3002', 'localhost:3003'];
const loadBalancer = new LoadBalancer(servers);
loadBalancer.start(3000);

## Popular Load Balancing Solutions

### 1. Software Load Balancers
- **Nginx**: High-performance web server and reverse proxy
- **HAProxy**: Reliable, high-performance load balancer
- **Apache HTTP Server**: With mod_proxy_balancer module

### 2. Cloud Load Balancers
- **AWS Application Load Balancer (ALB)**
- **Google Cloud Load Balancing**
- **Azure Load Balancer**

### 3. Hardware Load Balancers
- **F5 BIG-IP**
- **Citrix NetScaler**
- **Radware**

## Best Practices

1. **Monitor Server Health**: Implement comprehensive health checks
2. **Plan for Failover**: Ensure graceful handling of server failures
3. **Consider Geographic Distribution**: Use geographic load balancing for global applications
4. **Implement SSL Termination**: Offload SSL processing to the load balancer
5. **Log and Monitor**: Track performance metrics and request patterns
6. **Security**: Implement DDoS protection and rate limiting

## Conclusion

Load balancing is essential for building scalable, highly available systems. Choose the right algorithm and implementation based on your specific requirements:

- Use **Round Robin** for simple, equal-capacity servers
- Use **Weighted Round Robin** when servers have different capacities
- Use **Least Connections** for long-lived connections
- Use **IP Hash** when you need session persistence

Remember that load balancing is just one part of a comprehensive system design strategy. Combine it with other techniques like caching, database optimization, and microservices architecture for optimal results.