SysFlow

A cloud-native system telemetry framework that enables the creation of security analytics on a scalable, pluggable open-source platform.

Overview

SysFlow is a runtime observability framework designed to make security-related data science tasks easy. Its core is an open telemetry format that records how processes and containers interact with their environment, including the network, filesystem, and other processes. Its compact format enables the creation of stateful system behavioral graphs from streaming data, providing important context for security analysis.

SysFlow collects system events using the latest in eBPF technology to achieve portability across modern environments. The collection layer uses the CNCF Falco libraries to collect system events for downstream tasks, including real-time analysis through a stream analytics pipeline that accepts user-defined plugins, tags telemetry records with MITRE TTPs, and exports events to storage and analytic backends. Users can create their own machine learning analytics plugins, and chain them together using a Golang framework.

An integrated Jupyter environment makes it easy to explore and visualize the collected data. The supported Apache Avro schema enables users to import SysFlow using most major programming languages and toolkits with the C++, Python, and Golang APIs already available. Our new SysFlow lab is an excellent way to get familiar with the project.