I led the team that built Channel 4’s recommender system for All 4 in 2016. It started out as a straightforward project. But after getting lost in a rabbit hole trying to devise a score for ‘provocativeness’ and ‘serendipity’, I learned the single most important lesson about data science.
When they work, recommender systems feel almost like magic: based on these things you like, here are some other things you might also like.
The trend towards personalisation has been a major driver behind data science’s popularity, and with good reason. Netflix claims that algorithmically-generated recommendations account for 80% of views, and improved Google News clickthrough by 38%.
You might think the hardest part about building a recommender system would be the core mathematical algorithm itself. Not so. The hardest part about building the recommender system for Channel 4’s ‘All 4’ digital platform turned out to be knowing whether we were doing a good job. Now, knowing whether you're doing a good job is pretty important in any field, but it's especially critical in machine learning.
At its core, machine learning is about building algorithms that automatically improve at a task with experience. The word ‘improve’ is critical here. In order for the mathematics to do its work, you have to be able to tell the algorithm, with a number, how well it’s doing at every moment.
So, to build a recommender system, you need to be able to put a number on how good your recommendations are.
Surely that’s a straightforward thing to create?
Naïve first attempt: Average Hit Rate
A good recommender system should suggest things you want to watch. So let’s start with a simple approach, called ‘average hit rate’ in the lyrical vernacular of the field.
Based on what you have watched recently, the algorithm has to guess what you are going to watch next. ‘Average hit rate’ measures how often it guesses right.