Multiagent Learning under Strategic Behaviours Through the Lens of Optimisation

Optimisation has been the core of many machine learning (ML) problems. In particular, most of the standard ML techniques can be casted as searching for a minimum (or a maximum) of an objective function (e.g., empirical risk minimisation in offline ML, or regret minimisation in its online counterpart). With the rise of multi-agent learning paradigms, such as federated learning, self-play training (i.e., the agent learns by playing against itself), and multi-agent reinforcement learning, there has been a shift from minimisation problems to minimax optimisation in the recent years.  This shift was mainly influenced by the appearance of generative adversarial networks (GANs), which uses a two-player zero-sum game model to learn the underlying generative model of data (and in which one player aims to minimise an objective function, while the other is trying to counteract, hence the minimax manner). However, the way this minimax problem is solved is still collaborative, as in essence the “opponents” still help each other to converge to a stable solution as fast as possible.

While this collaborative multi-agent learning framework still has its interesting and difficult challenges (existence of convergence, stability, etc), it still cannot capture all the multi-agent learning settings, as it assumes (quasi) full cooperation between agents. In this talk, I will discuss a number of problem settings beyond this collaborative multi-agent learning framework, that allows agents to be selfish or strategic. The common thing in them is that agents don’t have to be fully cooperative anymore, but can follow strategic and selfish behaviours. These problems include: (i) last round/last iterate convergence in non-cooperative multi-agent learning; (ii) efficient learning with limited verifications against strategic manipulators; and (iii) truthful machine learning. Our work has been published at top tier AI/ML conferences such as NeurIPS, AAAI, AAMAS, ALT, and IJCAI.