Do you enjoy the movies that Netflix recommends to you? How often do you buy the products from the “others also bought” section? Countless businesses now use recommendation engines to improve customer experience and increase revenue. The cool thing about recommendations is that they’re awesome both for you as a business and you as a user. But why? Why are recommender systems beneficial for so many companies?

What is a recommender system?

A recommender system, or recommendation engine, is a data filtering tool that analyzes available data to make predictions about what a website user will be interested in. AI-powered recommendation engines are widely used in commercial applications, especially in e-commerce, social media, and content-based services. Based on the information the system has from users’ activity like what content they displayed, what products they bought together, etc., recommendation engines can accurately predict that a given user will be interested in a particular product thanks to the use of machine learning algorithms. To illustrate: services such as Netflix, Spotify, YouTube, Facebook, and Amazon use recommendation systems to predict which users will be interested in particular products or content, so whenever you see anything under “Recommended for you”, “Your playlist” or “Other shoppers also bought” – that’s the result of recommendation algorithms figuring out what stuff you like in order to increase the number of cross-sells and up-sells.

What are the types of recommendations?

There are 3 basic approaches to recommendation engines: collaborative filtering, content-based filtering, and hybrid recommendation systems.

Collaborative filtering

Collaborative filtering is based on the assumption that if users agreed in the past, they will also agree in the future – meaning that if they liked the same things previously, the situation in the future won’t change. This method requires collecting and analyzing information about customers’ behaviors, their activities, and their preferences to identify patterns and provide accurate predictions based on the similarity to other users. Let’s take a simple example to illustrate this approach: if John likes items A, B, C, and D, while Mike likes items A, B, and C, chances are he will also like item D.

However, collaborative filtering has some drawbacks. These issues include:

Early rater
When a new item appears, the system can’t recommend it because it doesn’t have any ratings for the item. It will take some time to get a sufficient number of ratings for the system to figure out what groups of users should be recommended the item.

Sparsity
With huge product bases, it’s difficult to make sure that enough people explore all the options available. If some item hasn’t been rated by a lot of people, the system won’t have data to base the predictions on.

Gray sheep
In order to recommend items, the system has to group people with overlapping interests. Many users will fall into these groups and enjoy the recommendations, but if some users don’t consistently agree or disagree with some group, they will not be given high-quality recommendations.

Content-based filtering

Content-based filtering focuses on the attributes or descriptive characteristics of items to generate product recommendations. In this approach, keywords are used to describe the item, and a user profile is built to show what kind of items the user likes. The assumption here is that if you expressed interest in some item, you will also like items with similar characteristics, be it the topic of an article, a brand of products, color, shape, size, etc. This approach is often used with recommendations of articles and other text documents.

Content-based filtering has some drawbacks as well:

Content description
In some cases, providing accurate descriptions of an item can be very difficult. If it’s music or videos that we’re trying to recommend, the representation of content is not always possible.

Over-specialization
If previous user behavior doesn’t show evidence for a user liking something, the system will not suggest it. If we want to system to provide recommendations outside the scope of what the user has already shown interest in, additional techniques need to be added.

Subjective domain
Content-based filtering techniques don’t deal well with subjective information such as point of view or humor.

Hybrid recommendation systems

As both of the approaches described above have some drawbacks, a solution was offered: to combine both approaches to deliver better results. And it worked – hybrid systems prove to be more effective. What are the risks related to using a single approach?

A hybrid recommendation system makes use of both the representation of content and the similarities between users. There are a few ways to implement hybrid systems: making collaborative and content-based predictions separately and combining them, by adding content-based capabilities to a collaborative approach or the other way round, or by unifying the approaches into one model. Netflix is an example of the hybrid approach, combining the habits of similar users (collaborative filtering) and similar characteristics to content previously likes by a user (content-based filtering) to provide awesome product recommendations.

What are the benefits of recommendation systems?

Recommendation engines can significantly increase revenue, improve CTRs and conversions. It also contributes to the improvement of factors more difficult to measure, such as customer satisfaction, and it increases customer retention.

Sounds cool, doesn’t it? After all, we know that users value personalized experience. In fact, 59% of shoppers who have experienced personalization say it has a big influence on their purchase decisions. With recommender systems, you get comprehensive insight into both your customer base and your product base. You can easily see how users interact with the service and generate reports. Recommenders find patterns in a blink of an eye to increase to the probability of the user finding an item of interest, thus cutting down on the time needed to find it. The personalized experience boosts customer satisfaction which translates into increased customer loyalty, increased consumption, and more profit. Additionally, with personalized content, such as newsletters, ads or notifications, you encourage users to come back to your site, so the number of visits goes up while churn drops.

It all looks wonderful, but let’s dig into some details. What benefits do recommendation engines bring to companies?

Personalized experience

In the past, personalized shopping experiences were a luxury available to the most affluent shoppers. A dedicated shopping assistant would recommend products and offer advice. Now, AI does that for us, and within milliseconds. Customers like this personalized touch – after all, most of us want to be recommended stuff: we ask family and friends about a variety of products and services. While their opinion matters, what works for them doesn’t have to work just as well for us. If you’ve never been disappointed by a product or service recommended by someone close to you, you are truly special. But recommendation engines do not have a taste of their own – it absorbs the user’s individual preferences to deliver the most relevant results. The process is similar: if your aunt likes movie A and movie B, and you also like movie A, you may as well like movie B. At this stage, before we go further, please note that this is largely simplified and does not reflect the actual operation of a recommender system – it would not be able to work on data about 2 people and 2 movies. But with data from hundreds of thousands of users and a wide variety of items, it rocks. And personalized experience is there to make the customer happy, which leads us to the next benefit…

Customer satisfaction

Never underestimate the power of making your customer happy. And the risk of making them mad! 90% of customers say that their purchase decisions are influenced by online reviews, while 86% say that their decisions are influenced by negative reviews. But! Customers who had a bad experience with a company are twice to three times more likely to post a review than those who were happy. And, as reported by Entrepreneur, even one negative review can cost a business 30 customers. With personalized shopping experience, however, you can actually make them happy. The system stores data about their recent activities. So let’s say I go online and I look for a green T-shirt. Then, I click through a few accessories. I leave the site just to go back to it because one of these accessories was a cool bag that I now want. But how the hell do I find it? Many sites “remember” what the user did in their last session and displays the recently displayed items. The collection of on-site interactions has one additional benefit: this data can be transferred to the offline, too. Burberry uses that to enhance the customer experience in their brick-and-mortar stores through analyzing customer data (e.g. purchase history) and providing relevant product recommendations that in-store assistants can then use to offer well-informed suggestions.

The “discovery” factor

Do you know Spotify’s “Discover Weekly” or Instagram’s discover feature? Both Spotify and Instagram analyze what content a user interacts with – played songs, watched videos, liked posts, favorite artists, frequently viewed profiles. This data gives away a lot of information about the user and just from that, the system can already figure out patterns and suggest relevant content that the user will likely want to follow or interact with. That’s great for two reasons:
1) users are overwhelmed with the amount of content available online and finding the things they are really interested in may not be that easy – especially when they’re looking for new “discoveries”, e.g. new bands or movies,

2) when users discover new, relevant content they are more likely to stay on the site longer and keep interacting with it – this may influence their decision on keeping a subscription (e.g. Spotify) or allow the platform to display new ads to the user when they’re online longer (e.g. on Instagram).

User engagement

Yet another benefit related to the personalization of online experience. Users tend to engage more with the available content when they’re served with it. It’s very logical when you think about it: it’s so convenient to click through “Related products” – it’s how we read articles, how we watch videos, how we shop. When a user has to search for each item separately, the chances of them giving up on the service go up. What’s more, with a recommendation engine, you can drive more traffic to your website through custom emails or ads. User engagement doesn’t have to end at on-site activity, you can nurture the shoppers to convert them into your customers. How do you do it? You bring them back to your site. You may do it with a personalized email as Zalando does. If a user added an item to cart and then removed it and left the site, in a few days, they may receive a message with a discount for this particular product. Automated messages about abandoned carts are already obvious to most online retailers.

Increased revenue

Personalized customer experience, increased customer satisfaction and engagement all lead to more revenue. How? There’s a number of ways. First of all, with the help of recommender systems, the shopper can find items they like without having to look for them. They added one item to the cart and were already about to check out, but then they saw a recommended product and went on browsing. This way, you increase the number of items per order and the average order value. If your service is subscription-based, you want your customers to stay with you and not leave for competition – to reduce churn, you personalize the offer. For telecoms, it means offering better deals for services that customers actually need and want or adding extra benefits – like a Netflix subscription at a lower price, or free of charge. For entertainment services, like Netflix or HBO Go, it means suggesting the best content to suit the customer’s needs, so they don’t feel like there’s nothing to do on the site. Being served with suggestions of what to watch, they feel taken care of. Whatever products or services you recommend, the goal is to reduce churn and increase the customer lifetime value. And it works – after implementing their recommendation system, Amazon reported a 29% increase in sales, while Netflix reports that 80% of watched content is based on algorithmic recommendations.

Making use of big data

The data of your customers is your most valuable possession – but only if you can make sense of this data and make sure it’s actionable. Otherwise, you’re left with strings of random information. To even consider building a recommender system, you must have relevant data that the system can utilize, and most services already collect this data, storing information about purchase history, search phrases, clicked items, etc. All of these don’t make sense for a human, unless you want to dedicate a group of people to analyze hundreds of columns of a spreadsheet in hopes of finding patterns. To be honest, it’s just outside of human capabilities and would be a great waste of time. A recommendation system, however, being fed with the right information, will produce great suggestions, enriching the customers’ profiles.

Does my business need a recommender system?

If you’re reading this, then chances are it does. E-commerce, entertainment, media, educational platforms – all of these can make great use of recommendation engines. In the modern world, with big players already using AI-powered solutions, it’s a must if you don’t want to stay behind. In order to be able to compete with other successful companies, you have to keep up the pace. The competitive advantage now lies in personalized customer experience, which cannot be achieved manually with a growing number of items and expanding customer base.

Articles by Michał Trojnarski

How to Boost Sales With a Recommender System?