Johnn.Nalwa
Log Shipper Sidecar Injection Webhook
all projects
cloud

Log Shipper Sidecar Injection Webhook

Kubernetes Mutating Admission Webhook in Go that automatically injects log-shipper sidecars into ML monitoring workloads — enabling centralized observability without manual pod spec changes.

GoKubernetesDockerFluent BitPrometheusYAMLMLOps

/// overview

Built a Kubernetes Mutating Admission Webhook written in Go that intercepts pod creation requests for ML monitoring workloads and automatically injects a Log Shipper sidecar container. The webhook enforces log collection, structured output, and deployment policy constraints across dynamically created ML jobs without requiring application-level changes.

The problem

ML monitoring workloads spin up dynamically and unpredictably. Manually instrumenting each pod for log collection was operationally unsustainable and produced observability blind spots when workloads were created by orchestrators rather than humans.

The approach

Implemented a TLS-enabled Mutating Admission Webhook server in Go, deployed as a Kubernetes service. On pod creation events matching target namespaces/labels, the webhook patches the PodSpec to inject a sidecar container running the configured log shipper (Fluent Bit / Logstash). Included admission policy enforcement logic and structured audit logging.

Outcome

Centralized log collection across all ML monitoring workloads with zero manual instrumentation. Reduced observability gaps and enabled consistent log shipping policy enforcement cluster-wide.