Recommender Systems: It’s More Than the Movies

This article is based on a speech delivered during Neoteric AI Talks Amsterdam on March 5th, 2020. Download this article as a PDF.

You may have already asked yourself this question: How do I empower and accelerate my business with AI? One of the most common, and successful, use cases among the biggest companies that have built a strong competitive edge is using recommender systems. 

How familiar are you with recommender systems and how they work? In case you need a recap: a recommender system is a kind of filtering tool. It goes through your data to find some specific information. The data that is analyzed is your customers’ data and product data. So the systems go through all this information and it generates recommendations: suggestions of items, like products or content, that a given customer will like.

Why does my business need a recommender system?

Alright, it all sounds good, but why would your business need a recommender system, too? See, there are various benefits that an organization can experience thanks to personalized recommendations. 

First, we can increase customer satisfaction level. How’s that possible? If we already have a recommender system in place and customers trust the recommendations, they will buy products that are recommended to them – and that they might not have bought otherwise. 

We can also significantly improve customer retention. Think about this situation: your customer finds your company and buys your product – one of many products from your assortment. After a while, your customers’ preferences are changing, and they’re no longer as interested in your offer as they used to be. A recommender system has this one interesting feature: it can calibrate the preferences that are changing in time. Thanks to this, we can retain customers for a longer time – because the recommender system is following their preferences and changing the recommendations accordingly. 

There are these sections called “Frequently bought together” or “If you liked this product, you will also like …”. Many online retailers have these. That’s the discovery feature – when customers can find products they will like without having to look for them. Such a practice improves the upsell process. This can be clearly explained with the use of a simple example: imagine you’re buying item A, let’s say a book. After adding it to cart, you see other recommended products, which can vary – it can be a book on a similar topic, it can be tea. It’s often a product you wouldn’t normally consider buying along with your chosen product. But then you see it and it makes sense – you get it as well. 

Last, but not least, we have something that’s a result of all the previous factors: increased revenue. When we increase the level of customer satisfaction and customer engagement, when people like getting recommendations, when they’re interested in your offer, they buy more – and do it more often. All that has an influence on revenue – making it grow.

Recommender systems in practice

Let’s have a look at some real-life examples. There’s Amazon. Amazon recommends products at virtually every step of the customer journey. It looks simple, right? But let’s have a look at the numbers: A year after implementing its recommendation engine, Amazon observed a 30% revenue growth. For a company as big as Amazon, this 30% translates into 30 billion dollars. 30 billion just because of a recommender system.

Another example: Netflix. I guess everybody knows what Netflix is, you may also know how it uses recommendations – telling us what movies and series we’ll enjoy watching next. A few years ago, McKinsey prepared a report stating that 80% of content watched on Netflix is watched only because of recommendations. This means that Netflix can promote 20% of their content and be fine – and not spend excessive amounts of money promoting every piece of content there is. According to the calculation McKinsey made, Netflix saves approximately 1 billion dollars a year on marketing efforts thanks to recommendations.

Let’s move on to Google. Google has many ways of utilizing recommender systems, but let’s only focus on two of them: first, autocomplete. When you’re typing something in the search box, you get predictions, suggestions of common phrases. These are not random – they’re based on your browsing history, location, current trends, and other factors – so these predictions are not the same for everyone. The second example is Google ads – targeting ads. Google knows exactly where we live, where we work, what our hobbies are. It knows everything it needs to know. The company can then use all this information to find ads that are most relevant to particular users. Google ads revenue from last year was about 130 billion dollars. Google built a strong competitive edge just by making smart use of their data. 

But recommender systems are not just for products. One of our clients, a major international financial institution, struggled with creating personalized career paths for their employees. They came up with an idea to create a platform that would allow employees to accelerate their professional development. However, this learning path should not only consider the employee’s goals but also company objectives. The learning path should match both requirements and meet all expectations. In the past, the process of creating career paths was very slow and manual, and they wanted to automate this process. To aid that, we built a recommender system – a custom machine learning algorithm that we trained on historical data to allow it to create personalized learning paths for the organization’s employees. The process was much faster once automated, and we could see a significant increase in employee retention.

Here’s another interesting case: one of the major Polish telecoms waned to reduce customer churn with AI. Creating a machine learning algorithm that was trained on their historical data, we were able to build a system that flags customers at high risk of leaving. What’s more, the recommender system created personalized offers that helped retain customers. Thanks to this, we managed to reduce churn by 24% without a negative impact on average revenue per customer. 

And one more case from the hospitality industry. One of our clients runs a business travel company and already had a recommender system in place for hotel recommendations, but it didn’t match their expectations. They wanted to improve their model, but instead, we built a new one. The new model turned out to be 10% better than the previous one.

How do we do it?

Start with outlining your recommendation strategy. Before we start, we need to know what kind of information the model should return. Is it one product? Many products? Should it explain why such recommendations are given? 

Then, focus on data. In supervised machine learning, data is a bottleneck. We have to collect it, organize it, clean it, do exploratory analysis. Then, we have to identify the similarities between users and products. The next step is to track user interactions. Measuring user interactions will allow calibrating the recommender system to user preferences that are changing over time. 

What’s important is that you should focus on a custom approach, a custom machine learning algorithm that will be suitable for your data. Even a small difference in data can have a huge impact on the model. 

Now that you know how recommender systems work, what benefits they bring, and where you can use them, there’s still one more thing you need to remember. It has been reported that 85% of AI projects fail to deliver. That can happen for a number of reasons: technical issues, business issues. But you have to remember that it’s an R&D project, so at the beginning, it’s difficult to estimate the right time and budget. Without knowing how long it will take and how much money it’ll consume, how can you move forward? On the one hand, we see the enormous potential of AI, and on the other hand, how do we know that it’s worth it? After all, if you fail, you waste time and money. 

To mitigate the risk of failure and boost your chances of successful AI adoption, we start any AI project with AI Sprint. The AI Sprint validates your data hypothesis before you jump into any costly projects. During the AI Sprint, you’ll get answers to questions like: Can my challenge be solved with AI? Can my goals be achieved with it?

The AI Sprint is a fixed time, fixed budget solution to validate your hypothesis. It starts with a 2-day workshop during which we dig into the detail of your business processes, define the business goal and metrics as well as technical success metrics. Then, we focus on data, analyze it, do feature engineering. We try out as many as possible machine learning algorithms that can potentially solve your problem and select the best one: the Proof of Concept. At the end of the AI Sprint, we prepare a detailed report, calculate the probability of success of full AI implementation in your organization, and provide recommendations for AI adoption. With AI, you need to start smart to win the game.