Back to All Events

How Spotify Analyzes Thousands of A/B Tests With Confidence

Abstract

Confidence (spotify-confidence on PyPi) is an open-source Python library that has been used to analyze experiments at Spotify for more than four years. It is now also an integrated part of Spotify's internal experimentation platform, where it's used to automatically analyze thousands of experiments every year. In this talk, we will discuss its benefits, demonstrate how to use it, and talk about what lies ahead.

Per Sillrén

Senior Data Scientist @ Spotify

Per Sillrén is a Senior Data Scientist and has been at Spotify in Gothenburg for almost five years. He works with product teams to understand user behavior and ensure product changes are evaluated using relevant metrics and scientific methods before being rolled out. He has a PhD in applied physics from Chalmers and also spent a few years implementing ML algorithms in ad tech before joining Spotify.