We address multi-agent Stackelberg settings involving many leaders and followers. In order to effectively model this kind of interactions, we extend the idea of commitment to correlated strategies to Stackelberg games with multiple leaders and followers. Correlation can be easily implemented by resorting to a device that sends signals to the players, and it also enables the leaders to reach better solutions than those achieved by committing independently. In this setting, a crucial question is how the leaders agree on a correlated-strategy commitment. To this end, we introduce a preliminary agreement stage that implements a natural non-cooperative negotiation protocol. The protocol proposes a correlated strategy to the leaders, who can then decide, in turn, whether to participate in the commitment or defect from it by losing the possibility of being part of the agreement. The goal is to design stable agreements in which no leader defects. We distinguish three solution concepts on the basis of the constraints that they enforce on the agreement reached by the leaders. We provide a comprehensive study of the properties of our solution concepts, in terms of existence, relation with other solution concepts, and computational complexity. As for the computational analysis, we prove that our solutions can be computed in polynomial time for certain classes of succinctly represented games. Interestingly, our results show that, in these games, introducing the agreement stage does not make computing equilibria a more challenging task, as finding a solution in our setting is as hard as computing an optimal correlated equilibrium.