Ever wondered how sites like Netflix are consistently able to recommend movies or shows you want to watch? Ever been on youtube and wondered how their recommended section works? According to Amazon, an astonishing 70% of their sales come through their recommendations alone. Not only that, over 80% of Netflix’s watch time is due to their recommended shows and movies.
Just like almost everything nowadays, we have machine learning to thank for that. Machine learning forms the backbone of all these recommendation systems in all these sites and products.
In this article, let’s try to explore how some of these recommendations work, and then we will move on to building one of our own.
While most sophisticated recommendation systems make use of Machine Learning algorithms, there are some which are based on just statistics alone.
In Netflix, one of the most basic ways of recommending movies is the ‘trending movies’, i.e. the movies which are being watched by the most number of people and based on the ratings. (Very simple, and no ML involved).
We could also use rule-based recommender systems. For example, my algorithm for recommending movies could be based on recommending movies of the same directors, or the same actors.
We will be going over the machine learning approaches to building recommender systems, which are of two types:
1. Content-based
As the name indicates, content-based filtering depends on the description of the product. Here, the recommender system generates a profile for the product, which serves as the attributes for comparison. Then, depending on the keywords, the recommendation engine correlates the product and the profile through a similarity score, which assigns a priority to the product. Finally, the product is displayed to the user based on the similarity scores against other products.
In the case of our movie example, the product is the movie and the attributes or keywords associated with it could be the title of the movie, description of the movie, the directors, the actors, etc.
2. Collaborative filtering
The limitation of just content-based filtering is that it recommends based on the similarity of items.
What if we could leverage the similarity between the users for recommending?
Collaborative Recommender systems (Social Recommender systems) are one of the most common engines that power various websites. Such recommendation engines analyze user behavior, preferences, activities, etc. to predict their likes or dislikes, depending on the insights drawn from similar users, so as to make relevant suggestions. Collaborative filtering can be user-to-user recommendations (user-based collaborative filtering) or item-to-item recommendations (item-based collaborative filtering).
3. Hybrid
A hybrid recommendation system is the best of both worlds approach that combines collaborative and content-based filtering to make the recommendation engine more effective. The combination of the two mitigates the drawbacks and amplifies the benefits of the pure approaches.
Netflix is a shining example of a hybrid recommender system that uses collaborative filtering (by processing the searching or viewing habits of the user) and content-based filtering (by suggesting content that shares common characteristics with what they have recently watched or rated).
While we have gone through some of the most common approaches being used in the industry for recommendation engines, this article would be incomplete without showing how you can build one of your own using Python and basic machine learning.
We are building a very simple recommender system based on content-based recommendations, using a movie dataset having 5000 entries. The following colab notebook contains the code and guided steps for the same:
https://colab.research.google.com/drive/1xgCa8h2kXS3EtL0pySBAMjHpRsSf9Vfv?usp=sharing
Hope this gives you a good starting point for building recommendation systems for your own use case. As a challenge, you can now learn about building a recommendation system based on collaborative filtering!
– By Shivangi Tomar, Third Year Department of Computer Science Engineering