Implementing eBPF-based Observability with Cilium 1.13 and Hubble 2.0 for Kubernetes Clusters: A Deep Dive

Introduction to eBPF and Cilium

When I first started working with Kubernetes, I realized that observability was a crucial aspect of managing and optimizing our clusters. Last quarter, our team discovered that traditional monitoring tools weren't sufficient for our needs, especially when dealing with complex network policies and security requirements. That's when we decided to explore eBPF-based observability with Cilium 1.13 and Hubble 2.0.

What is eBPF?

eBPF (extended Berkeley Packet Filter) is a technology that allows us to run sandboxed programs in the Linux kernel. It provides a powerful way to inspect and modify network traffic, making it an ideal solution for observability and security use cases. I was surprised to learn that eBPF is not just limited to network traffic; it can also be used for system calls, disk I/O, and more.

Cilium and Hubble Overview

Cilium is an open-source project that leverages eBPF to provide a robust and scalable networking solution for Kubernetes. It allows us to define network policies using a simple and intuitive API. Hubble, on the other hand, is a companion project that provides a unified observability platform for Cilium. It offers a real-time view of network traffic, security events, and system performance metrics.

Implementing eBPF-based Observability with Cilium and Hubble

To get started with eBPF-based observability, we need to install Cilium 1.13 and Hubble 2.0 on our Kubernetes cluster. The installation process is straightforward, and the official documentation provides a step-by-step guide. Once installed, we can define network policies using Cilium's API and monitor them using Hubble.

Step 1: Install Cilium and Hubble

helm install cilium cilium/cilium --version 1.13
helm install hubble hubble/hubble --version 2.0

Output:

NAME: cilium
LAST DEPLOYED: Mon Jan 15 10:23:45 2024
NAMESPACE: default
STATUS: deployed
...

Step 2: Define Network Policies

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: allow-https
spec:
  endpointSelector:
    matchLabels:
      app: web
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: client
    toPorts:
    - ports:
      - 443

Step 3: Monitor Network Traffic with Hubble

hubble observe -n default

Output:

TIME                  SOURCE IP       DEST IP       PROTOCOL   PORT
2024-01-15T10:23:45Z  10.0.0.1       10.0.0.2       TCP       443
2024-01-15T10:23:46Z  10.0.0.2       10.0.0.1       TCP       443
...

Conclusion

Implementing eBPF-based observability with Cilium 1.13 and Hubble 2.0 has been a game-changer for our team. We can now monitor and optimize our Kubernetes clusters with unprecedented granularity and precision. If you're struggling with observability in your Kubernetes environment, I highly recommend exploring Cilium and Hubble.