the algorithm works

I was driving in the rain when a song called Cool Estatitude came on shuffle. It was sandwiched between something bible belt-y and a headbanger called Hard Everything by Spookie Ed on my Discover Weekly playlist.

Reduced down to 1hr and 36m of music I felt incompletely and incorrectly defined, and SOL for finding new music for an indoor cycling class less than an hour away. And it was all my fault.

Where did I go wrong, how do you correct an algorithm run awry, and what do you learn from said correction?

how Spotify’s Discover Weekly algorithm works

Every Monday, Discover weekly extends our horizons by 30 songs, shipping users off into the long tail waves of new musical worlds.

To make your personalized Discover Weekly playlist, Spotify builds a taste profile based on your listening habits, then uses collaborative filtering to find songs you haven’t listened to from people with similar taste profiles. It’s the same approach that helps you buy more stuff on Amazon!

For example:

  • you like Maggie Rogers, Drake, Odesza and the xx

  • some cool stranger likes Bishop Briggs, Drake, Odesza and Alt-J

Spotify sees your similar tastes and constructs the next Discover Weekly playlist so that:

  • yours features Bishop Briggs and Alt-J

  • some cool stranger’s features Maggie Rogers and the xx

Well, kind of. In addition to collaborative filtering, Spotify also uses:

  • natural language processing, which uses text to tag every song and artist with thousands of top terms identify similarities between 

  • raw audio analysis, which reviews key characteristics of every song to identify similarities and increase the discoverability of new music (that might be missed in the first 2 models).

how I messed up Spotify’s algorithm

Spotify knows you pretty well. So…how did I mess up their algorithm?

Enter, Soundcloud.

Soundcloud offers over 200 million songs, including millions of unauthorized remixes and mashups made for indoor cycling. I treasure hunt once a week, armed with time, patience and AirPods to search for audio gold.  

Thanks to Soundcloud, 77% of the songs on my cycling playlists consist of local files, a feature that allows me to bring music beyond the 50 million songs on Spotify into the Spotify galaxy. The remaining 23% of songs comes from Spotify in the form of random house  and obscure cool down music.

Local files exist in my personal but public black hole. Spotify simultaneously acknowledges and ignores my local files. Imported songs aren’t analyzed to recommend related music. To Spotify’s algorithm, I am what it sees: the cool down music and arms songs that seem to be all that I add to playlist after playlist.

My Discover Weekly playlist had me blasting Cool Estatitude while questioning my sanity because the majority of my music is either consumed outside of the system or invisible to Spotify.

how do you fix broken algorithms

Rather than trying to understand the problem I usually text a friend to complain. My default reaction is that Spotify doesn’t get me, the algorithm doesn’t work.

In reality, Spotify doesn’t get me for the same reasons we don’t get each other: the algorithm works off a limited and incorrect set of data about me and “people like me.”

Luckily, Spotify’s algorithm is great, and fairly easy to retrain. You can brute force your way to better Discover Weekly playlists by skipping music you don’t like, playing new music, and putting music you love into playlists. Easy peasy back to cheesy.

It’s not so simple in life. Most of the work we do, experiences we go through and feelings we have are invisible to other people. The algorithms that run our everyday do so with an incomplete picture of who we are, pulling highlights from the most polished public channels. Most of the algorithms are not as logical as Spotify’s. We all get reduced down to shitty Discover Weekly playlists.

In the last month, have you:

  • been misunderstood by a coworker?

  • felt overlooked by a friend or significant other?

  • had a project go totally sideways?

  • been put into a clearly marked pigeon hole shaped box?

To change the algorithm, to reprogram it, you have to give it feedback. On Spotify as in life, feedback is delivered through action. When you act incongruently to the algorithm’s assumptions consistently enough, the algorithm changes.

It’s simple, but it isn’t easy. Direct feedback is hard to give and hard to take. Giving direct feedback persistently is challenging, too. Maybe Cool Estatitude isn’t so bad, after all?

still undefined

still undefined

Cool Estatitude kind of sucks. The algorithm should constantly change because you are constantly changing. The more you’re exposed to, the more you evolve. Learning agility starts with you, not the algorithm. 

For the system that supports you to evolve with you, you have to provide feedback. The output is only as good as the input. As you change, the algorithms change to include better, more diversified results. The system changes.

At least, in the case of Spotify. My Discover Weekly is on the mend, and thank goodness. I cut down on the time I spend on Soundcloud because, regardless of feedback, their discovery algorithm is terrible. 2020: the year I let go of algorithms that don’t serve me.