Talk (Work in Progress): Ignacio Ojea Quintana (MCMP)

Title:

The Reward Puzzle in Recommender Systems (with Silvia Milano)

Abstract:

Recommender Systems (RS) are an ubiquitous technology affecting consumption, social relations, news information and many other important aspects of our lives. They are usually justified as a technology that identifies and maximizes users’ preferences, by suggesting items (news, products, people) that bring the most utility to them. More generally, they are conceived as estimating a stationary distribution associated with the users’ preferences. We believe this conceptualization of users’ utilities or rewards is misguided, and we build on the ‘reward paradox’ in reinforcement learning to illustrate the problem: the utility that a user gets from an item depends not only on exogenous variables like the item and the context in which is suggested, but also endogenous variables like mental, cognitive, or other inner states. We then model the RS task as an interaction between two agents, a recommender and a user agent. This allows us to provide more psychological depth into the user agent, and we will show some preliminary results in that direction.