Optimized Ranking-Based Techniques for Improving Aggregate Recommendation Diversity

Master's Thesis from the year 2013 in the subject Computer Science - Miscellaneous grade: 1 course: ME computer science language: English abstract: This paper investigates how demand-side factors contribute to the Internet's Long Tail phenomenon. It first models how a reduction in search costs will affect the concentration in product sales. Then by analyzing data collected from a multi-channel retailing company it provides empirical evidence that the Internet channel exhibits a significantly less concentrated sales distribution when compared with traditional channels. The difference in the sales distribution is highly significant even after controlling for consumer differences. Furthermore the effect is particularly strong for individuals with more prior experience using the Internet channel. We find evidence that Internet purchases made by consumers with prior Internet experience are more skewed toward obscure products compared with consumers who have no such experience. We observe the opposite outcome when comparing purchases by the same consumers through the catalog channel. If the relationships we uncover persist the underlying trends in technology and search costs portend an ongoing shift in the distribution of product sales. Singular Value Decomposition (SVD) together with the Expectation-Maximization (EM) procedure can be used to find a low-dimension model that maximizes the log likelihood of observed ratings in recommendation systems. However the computational cost of this approach is a major concern since each iteration of the EM algorithm requires a new SVD computation. We present a novel algorithm that incorporates SVD approximation into the EM procedure to reduce the overall computational cost while maintaining accurate predictions. Furthermore we propose a new framework for collaborating filtering in distributed recommendation systems that allows users to maintain their own rating profiles for privacy. We conduct offline and online t