While LLMs and deep learning are all the rage nowadays, many problems still make some of these models infeasible. In cases such as network filtering, real-time security monitoring, etc., where one might need to make a decision within a few nanoseconds, there is no time to copy data to a userspace process, let alone to a GPU. This talk will outline work on building ML models to run in Linux kernel space, in particular, the eBPF virtual machine and the restrictions imposed on running models there. We will use a fairly naive dataset from Kaggle for malicious traffic detection and showcase how a model can be trained, compiled, and deployed in a real-life kernel without restarting the machine.
eBPF in and of itself poses a set of interesting constraints:
What you get in return is a formally-verified program (and model) that is guaranteed to be safe w.r.t., e.g. out-of-bound reads, predictable runtime complexity, and more importantly, a program you can deploy on any Linux machine with a fairly small set of capabilities without having to modify the running kernel.
The talk is aimed at both ML Engineers, developers and a little bit of SecOps, where I will outline the task and dataset with some flashbacks of caveats in traditional feature-engineering, in particular for (network) packet inspection. There might be lessons on how NOT to build datasets and on the importance of domain expertise in both dataset creation and feature engineering. Additionally, Jesper will give an introduction to eBPF and kernel development, and share lessons learned from building code (and models) that is portable across eBPF while still maintaining backwards compatibility with the original embedded-C targets.
In some sense, this is an ode to classical ML and to how traditional feature engineering still has a place, even if it's a small one.
This work is not sponsored by any company; it is open-source and in the public domain.
Jesper is a freelancing Tech Lead and Senior Machine Learning Engineer with a career spanning through AI/ML, software development, architecture, and research. His journey in machine learning spans more than 20 years, from developing an embedded computer-vision system to more recently internet-scale infrastructure, where roughly 30% of internet traffic flows through his models. Over the years, his career has spanned everything from natural-language processing to reinforcement learning (and everything in between). He is also an active contributor to many open-source projects, including the Linux kernel.
