Why do the we care about the Recommendation Systems, the answer to this question may be different based on different perspective. For example, for companies like Amazon, Spotify and Netflix is to generate more and more revenues and drive a significant amount of engagement to their websites that results in an exponential growth in their marketplace. But, for people using Amazon, Spotify and Netflix, it means saving their time and getting the things of their interest and those which are being highly liked into their suggestions, so that they don’t have to search for it. How cool is that…? this is the essence of Recommendation Systems or Recommendation Engines.
Recommendation Systems brings out the value and provide an amazing experience of personalizing contents for the users in cases where there are a lot of items.
One more interesting thing that why Data Analysts or Data Scientists must specifically care about the Rec-Sys, because it is the true Data Science because Recommendation Systems lies at the intersection of Machine Learning, Software Engineering and Statistics. And, for building a strong and a powerful Recommendation Engine, all of these skills are highly required.
Now, when it comes to define that what a Recommendation System is, there are a number of definitions that you can get, but in a single line and in the most-easiest way, Recommendation System can be defined as a system or platform that helps the users discover items or products they might like.
There might be different types of approaches for Recommender Systems, and those approaches are more practical than academic, because in real world there are a different types of complex Rec-Sys exploiting hybrid or ensemble methods of Recommendation approaches. But, we will see the common and basic types of approaches of Recommendation System.
Conceptually Recommender Systems or Recommendation Engines use three types of recommendation approach (or approaches).
- Collaborative filtering (CF),
- Content-based filtering (CBF) or
- Knowledge-based filtering (KBF).
It is perhaps the most common and widely used recommendation technique. In Collaborative filtering, the approach is to use the ratings of other users that who are similar to a user in the past to produce recommendations of items or products. For example, if you and your friend listened to the same music or watched the same movies, and your friend bought a new book that you do not know about, you might like the book as well. This approach requires more information about the users, that is the reason why at some places it isn’t employed, but for example, at Social Media websites like Facebook and Twitter, it is highly exploited individually. So this is the major limitation of Collaborating filtering because it relies on the preferences and information to get store, so in a cold- start scenario, it is not used, because there are no preferences at all, it won’t generate any recommendations. But, it can be employed with other approaches to make a very powerful recommender system.
In Content-based filtering, the approach is to create a profile for each product or item to characterize its nature or technically the category and Products or items which are having similar characteristics to the ones we liked in the past are recommended to us. For example, if you liked Sci-Fi novels in the past, a what Content-based recommender system will do, it will recommend you other Sci-Fi novels. This technique often requires more information about the items.
One of the main advantage of these Content-based Recommender Systems over Collaborative Filtering Recommender Systems is that it doesn’t require that much user preferences and information to work and even with a little known user preference and information it can generate quite good recommendations and bring the value to the situation.
In Knowledge- based Approach there is the need of using external information or knowledge to filter products based on their attributes. So, the Knowledge- based Recommender System recommend items and products based on a specific domain or category. These Recommender Systems relies on the situation about how much certain item or a product feature covers the user preferences and interests.
There is also a fourth approach that is also known as Hybrid Recommendation which inculcates any two or more of the above mentioned approaches of Recommendation Systems. And, as previously discussed, this approach can solve the problems like as the “cold start “problem, when enough information is not stored about the user’s preferences and interests.
Amazon Recommender System- How it works?
Now, let us move ahead and discuss the working of some practical and real world Recommendation System with which we all are familiar. The Recommendation System by Amazon, it is considered to be one of the best recommendation engine. The question is what technique and approach goes in behind the Amazon’s recommendation Engine? Being one of the best recommendation engine currently in the market, we will see how do Amazon provides such relevant and true recommendations to its users?
Recommending or suggesting something may be considered as both an art and a science. To build a recommendation system, it totally depends on the purpose that you want to serve. For different purpose, different approach may be used. Considering the Recommendation System of Amazon, the major thing revolves around market basket analysis (which is also called as Affinity Analysis), subset of the field of data mining.
The typical factors of such a Recommendation system include many considerations and the identification of various primary items and also the identification of the affinity items or products such as upsell and cross sell. The following are the various considerations that Amazon has to deal with for all of its users. It stores all of the data from each and every user that visits that website for every point of time.
- The user history is stored in the following classes.
- Purchased shopping carts, as this is the real money from real people that is spent on real items and it is the most powerful data for making primary recommendations.
- The data is also stored for the items or products that were added to carts but abandoned.
- There is also the need of Pricing experiments online (i.e. A/B testing, etc.) where they offer the same products at different prices and check the results.
- The data is also stored for every user’s wish lists – it contains the items or products that a user is going to buy, but is still looking for other similar items. This approach can be treated similarly to another stream of market basket analysis.
- The data is also stored for the information of the Referral sites (i.e. identification of information from where you came in and that is also a level of information to identify the interests or preferences of the user.)
- The data is also stored for your dwelling time or how long you checked any product or item before you click again and chose a different item or product.
- The data is stored for the number of times you viewed an item before buying it.
- It also stores information for the ratings that you provide to the items and also for those in your social media network or buying circles – if you rate things that you like you get more of other similar items based on that.
- The data is also stored for the demographic information of the user such as the shipping address which comprises to your city, state and country so that user can also get what is popular in his/her location.
- There is also the user segmentation, for example based on your profile if you buy 3 books in separate months for a child, that mean you likely have a kid. And, eventually, your recommendations will also include that.
- Amazon also promotes the direct marketing that is click through data, for example you might have got a number of emails from them and suggesting you to buy any particular product or item by click through. And, the information is stored if you bought that item.
So, in the above article, we started from understanding with a brief explanation of what a Recommendation System is? Why should we care about it? What are the different approaches of building a Recommender System and what are the most common types of present Recommender Systems? and how does it work? Next, we also understood, what happens behind the screens of one of the most efficient Recommendation System of today, the Recommender System of Amazon. So, this was the theory and concepts behind the Recommender System, in another article on the Recommender Systems, we will see how do we build a Recommender System from scratch using Python.