The Client operates in the Smart TV digital home entertainment market. It’s a leading premium video-on-demand service, which allows users to watch newly released movies in perfect quality or choose from a library of more than 7000 titles. Their SmartTV application has 1.5 million monthly active users.
When the Client went searching for a data science partner, 80% of the company’s revenue had already been coming from their recommendation page. The recommendations list had been updated by an editor every week.
In order to improve customer experience and drive more sales from the recommendation page, the Client decided to build a recommendation engine that would offer up-to-date personalized movie recommendations for every user.
The main goals of the project were:
InData Labs built a custom recommendation engine for the Client, to ensure that the system integrates perfectly with the existing business and technical environment of the Client’s Smart TV application.
InData Labs used machine learning to build a recommendation engine that serves individual recommendations for millions of users.
The recommendation engine uses a collaborative filtering approach which is based on the idea that people who agreed in their evaluation of certain movies in the past are likely to agree again in the future.
In order to generate fresh recommendations every day, the system is optimized to store data efficiently and perform queries faster.
It is well-known that columnar storage saves both time and space when it comes to big data processing. For this project, InData Labs used Parquet + Spark, which helped to boost Spark SQL performance by 5X compared to using other storage formats.