Smart Network Observability - Connection Tracking

Abstract

Flow Logs Pipeline (a.k.a. FLP) is an observability tool that consumes flow logs from various inputs, transforms them and exports logs to Loki and / or time series metrics to Prometheus. While flow logs encompass a lot of valuable data, observing the network from the level of flow logs is often too low. In many cases, we are interested in observing it from a higher level, the level of connections. In this work, we introduce a new processing stage in FLP that allows aggregating flow logs from the same connection - connection tracking.