If you use Netflix, Spotify, Amazon or almost anything online, you give and take from recommender systems every day. But are they more useful for particular people? Is there anything to gain if you’re already an expert? And how does the future look?
I’m an uncle to two babies. I live further away than I’d like from my family, so throwing presents in their general direction is the easiest way to my new tiny relatives’ hearts.
But what do you buy for someone that doesn’t know what an egg is? I don’t know if a 1-year-old would prefer Lego, a sticker book, a spice rack or a set of drums for christmas.
I did get a word of mouth recommendation to buy an Aquamat, where the kids can draw with water-filled pens without making a mess. A bit like an etch a sketch, but bigger, and more hands on.
That was my entry point. Now, thanks to Amazon’s recommender system, I know of a hundred things I can buy. As a novice on kids’ toys, I’m delighted with Amazon’s recommendations. It took me no research. Other mums, dads uncles and aunts did it for me, and I harvested the results from their data. Via Amazon. A novice’s dream.
Toy recommendations impressed me because I am a novice. But what if I’m an expert?
My mum bought me a deck and some records when I was thirteen. Ever since I’ve been collecting and consuming music like water. I’m an armchair expert in Bossa Nova. I won’t be a guest on a BBC 4 documentary but I can sing every word of almost every Astrud Gilberto song, in broken, (read: a nonsensical attempt at) Portuguese. I also love Psychedelic Rock.
Spotify’s Discover Weekly playlist has been a revelation for me. It showed what I didn’t know I didn’t know, that I should know. A group called Novos Baianos, recognisable by their Bossa Nova and psychedelic rock influences.
The discovery did two things: provided me new music to love, and improved my trust in Spotify’s recommendations. We have a symbiotic relationship. Now every week I listen to my Discover Weekly playlist and use the radio function, as it adds value for me.
But that wasn’t the most impressive part of it. My inroads into Bossa Nova began when I was listening to Runnin by Pharcyde. I loved the sample used, which you can find below. I started digging through my favourite songs containing samples for their originals. A treasure trove of new music.
When I got on Spotify and made a playlist of samples, Spotify started recommending other samples, like:
• The Edge - David McCallum. A Scottish classical, avant-garde (sorry) musician and actor. The sample from this is in The Next Episode by Dr Dre.
• Saudade Vem Correndo - Stan Getz, Luiz Bonfa and Maria Toledo. A Brasilian Bossa Nova track sampled in Runnin by the Pharcyde.
• Black Crow - Steely Dan. Jazzy pop rock sampled in Gas Drawls by MF DOOM.
Samples span eras, and genres, and often have nothing in common with each other aside from being samples. The above is a testament to that. But what's remarkable is that Spotify would still only recommend other samples for that playlist. The amount of new music I find per week that I love is much higher than pre-Spotify.
Recommender systems aren’t perfect. But then, if it was up to me, I’d recommend Preta Pretinha by Novos Baianos. 10% of people may love it. So my recommendation would score a much lower hit rate than the algorithm. And the algorithm will only get more accurate.
So recommendations can work for a self-styled 'expert', as they can for novices. But for every gem I’ve had, there have been recommendations that miss the mark.
If AI learns and evolves, that will eventually cease to be the case. So, it’s job done for the people at Spotify, Amazon or Netflix, right?
Well, no.
Netflix ran a competition to beat their own algorithm. The prize was $1,000,000, and in 2009 team ‘Pragmatic Chaos’ won it. Netflix felt it improved their algorithm by ten percent.
But they chose not to use it.
The engineering cost wasn't worth it. But more importantly, by the time they could have implemented it, Netflix felt they'd moved to the next level. That's $1,000,000, for what?
Netflix wants to know what can be done, even if they choose not to do it. They want their algorithm stacks to be smarter than anyone else’s. Because they want to continue scaling their business with future-proof strategy. And AI-driven recommendations are the future. For novice, expert and everything in between.
-----------------------------------------------------------------------------
Our CEO here at Filtered recently published a great post (no bias attached) on how human intelligence will evolve over the next 10 years. Moving those unknown-unknowns into being known-knowns. Click here to read more about human intelligence.