Research

05 Mar 21

Policy Optimization as Online Learning with Mediator Feedback

Policy Optimization as Online Learning with Mediator Feedback Authors: Alberto Maria Metelli, Matteo Papini, Pierluca D’Oro, Marcello Restelli Conference: AAAI 2021 Abstract: Policy Optimization (PO) is a widely used approach to address continuous control tasks. In this paper, we introduce the notion of mediator feedback that frames PO as an online learning problem over the […]

05 Mar 21

Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimate

Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimate Authors: Mirco Mutti, Lorenzo Pratissoli, Marcello Restelli Conference: AAAI 2021 Abstract: In a reward-free environment, what is a suitable intrinsic objective for an agent to pursue so that it can learn an optimal task-agnostic exploration policy? In this paper, we argue that the entropy […]

05 Mar 21

Newton Optimization on Helmholtz Decomposition for Continuous Games

Newton Optimization on Helmholtz Decomposition for Continuous Games Authors: Giorgia Ramponi, Marcello Restelli Conference: AAAI 2021 Abstract: Many learning problems involve multiple agents optimizing different interactive functions. In these problems, the standard policy gradient algorithms fail due to the non-stationarity of the setting and the different interests of each agent. In fact, algorithms must take […]

15 Feb 21

Learning Probably Approximately Correct Maximin Strategies in Games with Infinite Strategy Spaces

Learning Probably Approximately Correct Maximin Strategies in Games with Infinite Strategy Spaces Authors: Alberto Marchesi, Francesco Trovò, Nicola Gatti Conference: AAAI 2021 Abstract: We tackle the problem of learning equilibria in simulationbased games. In such games, the players’ utility functions cannot be described analytically, as they are given through a black-box simulator that can be […]

15 Feb 21

Online Learning in Non-Cooperative Configurable Markov Decision Process

Online Learning in Non-Cooperative Configurable Markov Decision Process Authors: Giorgia Ramponi, Alberto Maria Metelli, Alessandro Concetti, Marcello Restelli Conference: AAAI 2021 Abstract: In the Configurable Markov Decision Processes there are two entities, a Reinforcement Learning agent and a configurator which can modify some parameters of the environment to improve the performance of the agent. What […]

15 Feb 21

Inverse Reinforcement Learning from a Gradient-based Learner

Inverse Reinforcement Learning from a Gradient-based Learner Authors: Giorgia Ramponi, Gianluca Drappo, Marcello Restelli Conference: NeurIPS 2020 Abstract: Inverse Reinforcement Learning addresses the problem of inferring an expert’s reward function from demonstrations. However, in many applications, we not only have access to the expert’s near-optimal behaviour, but we also observe part of her learning process. […]

15 Feb 21

An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits

An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits Authors: Andrea Tirinzoni, Matteo Pirotta, Marcello Restelli, Alessandro Lazaric Conference: NeurIPS 2020 Abstract: In the contextual linear bandit setting, algorithms built on the optimism principle fail to exploit the structure of the problem and have been shown to be asymptotically suboptimal. In this paper, we […]

15 Feb 21

No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium

No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium Authors: Andrea Celli, Alberto Marchesi, Gabriele Farina, Nicola Gatti Conference: NeurIPS 2020 Abstract: The existence of simple, uncoupled no-regret dynamics that converge to correlated equilibria in normal-form games is a celebrated result in the theory of multi-agent systems. Specifically, it has been known for more than 20 years […]

15 Feb 21

Online Bayesian Persuasion

Online Bayesian Persuasion Authors: Matteo Castiglioni, Andrea Celli, Alberto Marchesi, Nicola Gatti Conference: NeurIPS 2020 Abstract: In Bayesian persuasion, an informed sender has to design a signaling scheme that discloses the right amount of information so as to influence the behavior of a self-interested receiver. This kind of strategic interaction is ubiquitous in real economic […]

15 Feb 21

Sequential transfer in reinforcement learning with a generative model

Sequential transfer in reinforcement learning with a generative model Authors: Andrea Tirinzoni, Riccardo Poiani, Marcello Restelli Conference: ICML 2020 Abstract: We are interested in how to design reinforcement learning agents that provably reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones. The availability of solutions to related problems poses a […]