Session based recommender system with multistage candidate sampling


DOI: 10.34759/trd-2022-126-20

Аuthors

Mokhov A. I.1*, Kislinskiy V. G.2**, Alekseychuk A. S.1

1. Moscow Aviation Institute (National Research University), 4, Volokolamskoe shosse, Moscow, А-80, GSP-3, 125993, Russia
2. Moscow Institute of Physics and Technology (National Research University), 9, Institutskiy per., Dolgoprudny, Moscow region, 141701, Russia

*e-mail: AIMokhov@mai.ru
**e-mail: kislinskiy.vg@phystech.edu

Abstract

Understanding users’ preferences is a challenging task especially with a huge amount of items. Modern recommender systems are keen to solve this task by applying state-of-the-art methods of candidate sampling and simple heuristics in couple with Machine Learning ranking algorithms. This paper presents an algorithm of candidate sampling from three different sources followed by a ranking algorithm. These two stages form a session-based recommender system that is capable of building a user’s probable preferences based on its current session. For candidate sampling, we use a language model (Word2Vec) and sparse vectors for item representations, and the most popular items from a dataset. Each stage is divided into multiple substages making it really simple to add new candidate sources or remove existing ones. The same technique can be easily applied to ranking algorithms — one can remove a ranking algorithm or add the new one in order to blend model predictions maximizing Precision or Recall metrics as well. We also show the importance of ranking algorithms in recommender systems by measuring Learning to Rank (L2R) specific metrics on test data. There are several ranking algorithms in this paper. All of them belong to the pairwise algorithms subclass. Such algorithms as LambdaRank, YetiRank, and StochasticRank are used in comparison to non-ranked recommendations. We use CatBoost implementation of gradient boosting and PyTorch to build a neural ranking net. As a result of the experiment, we get a ready end-to-end recommender system pipeline with flexible modules that are easy to add/remove and show the benefits of ranking models with recommendations on real data.

Keywords:

learning to rank, recommender system, machine learning, ranking metrics, CatBoost

References

  1. Burges C., Shaked T., Renshaw E., Lazier A., Deeds M., Hamilton N., Hullender G. Learning to rank using gradient descent, 22nd International Conference on Machine learning, 2005, pp. 89-96. DOI: 10.1145/1102351.1102363
  2. Burges C.J. From ranknet to lambdarank to lambdamart: An overview, Learning, 2010, vol. 11, no. 81. URL: https://www.microsoft.com/en-us/research/publication/from-ranknet-to-lambdarank-to-lambdamart-an-overview/
  3. Burges C., Ragno R., Le Q. Learning to rank with nonsmooth cost function, Advances in neural information processing systems, 2006, vol. 19, pp. 193-200. DOI: 10.5555/2976456.2976481
  4. Croft W.B., Metzler D., Strohman T. Search engines: Information retrieval in practice, Reading: Addison-Wesley, 2010, vol. 520, pp. 131-141. URL: https://www.isi.edu/people/metzler/publications/search_engines_information_retrieval_practice
  5. Devlin J., Chang M.W., Lee K., Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018, vol. 1, pp. 4171-4186. DOI: 10.18653/v1/N19-1423
  6. Prokhorenkova L., Gusev G., Vorobev A., Dorogush A., Gulin A. CatBoost: unbiased boosting with categorical features, Advances in Neural Information Processing Systems 31 (NeurIPS 2018), 2008, pp. 6638-6648. URL: https://arxiv.org/pdf/1706.09516.pdf
  7. Grover, A., Wang, E., Zweig, A., and Ermon, S. Stochastic optimization of sorting networks via continuous relaxations, Proceedings of the International Conference on Learning Representations, 2019. URL: https://arxiv.org/pdf/1903.08850.pdf
  8. Gulin A., Kuralenok I., Pavlov D. Winning the transfer learning track of Yahoo!’s learning to rank challenge with yetirank, Learning to Rank Challenge, 2011, pp. 63-76. DOI: 10.5555/3045754.3045761
  9. Koren Y. Factorization meets the neighborhood: a multifaceted collaborative filtering model, 14th ACM SIGKDD international conference on Knowledge discovery and data mining, 2008, pp. 426-434. DOI: 10.1145/1401890.1401944
  10. Koren Y., Bell R., Volinsky C. Matrix factorization techniques for recommender systems, Computer, 2009, vol. 42 (8), pp. 30-37. DOI: 10.1109/MC.2009.263
  11. Li H. A short introduction to learning to rank, IEICE Transactions on Information and Systems, 2011, vol. 94 (10), pp. 1854-1862. DOI: 10.1587/transinf.E94.D.1854
  12. Liu T.Y. Learning to rank for information retrieval, Foundations and Trends® in Information Retrieval, 2009, vol. 3, no. 3, pp 225-331. DOI: 10.1561/1500000016
  13. Mikolov T., Chen K., Corrado G., Dean J. Efficient estimation of word representations in vector space, In 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013.
  14. Felix Petersen, Christian Borgelt, Hilde Kuehne, Oliver Deussen. Learning with algorithmic Supervision via Continuous Relaxations, Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021). URL: esearch.ibm.com/publications/learning-with-algorithmic-supervision-via-continuous-relaxations
  15. Robertson S., Zaragoza H. The probabilistic relevance framework: BM25 and beyond, Now Publishers, Inc. 2009. DOI: 10.1561/1500000019
  16. Sun F., Liu J., Wu J., Pei C., Lin X., Ou W., Jiang P. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer, 28th ACM international conference on information and knowledge management, 2019, pp. 1441-1450. DOI: 10.1145/3357384.3357895
  17. Sun Z., Yu D., Fang H., Yang J., Qu X., Zhang J., Geng C. Are we evaluating rigorously? Benchmarking recommendation for reproducible evaluation and fair comparison, In Fourteenth ACM Conference on Recommender Systems, 2020, pp. 23-32, DOI: 10.1145/3383313.3412489
  18. Ustimenko A., Prokhorenkova, L. StochasticRank: Global Optimization of Scale-Free Discrete Functions, International Conference on Machine Learning, 2020, pp. 9669-9679, URL: https://proceedings.mlr.press/v119/ustimenko20a.html
  19. Ustimenko A., Prokhorenkova L. SGLB: Stochastic Gradient Langevin Boosting, International Conference on Machine Learning, 2021, pp. 10487-10496. URL: https://deepai.org/publication/sglb-stochastic-gradient-langevin-boosting
  20. Protsenko P.A., Khubbiev R.V. Trudy MAI, 2021, no. 119. URL: http://trudymai.ru/eng/published.php?ID=159756. DOI: 10.34759/trd-2021-119-18
  21. Smerchinskaya S.O., Yashina N.P. Trudy MAI, 2015, no. 84. URL: http://trudymai.ru/eng/published.php?ID=62973
  22. Zakovryashin A.I. Trudy MAI, 2012, no. 61. URL: http://trudymai.ru/eng/published.php?ID=35494

Download

mai.ru — informational site MAI

Copyright © 2000-2024 by MAI

 Вход