DRIVE is a decentralized peer-incentivization mechanism for emergent cooperation under changing rewards. Agents exchange local reward differences rather than fixed tokens, allowing incentive magnitudes to adapt automatically when environmental rewards are shifted, scaled, or perturbed.

The repository contains the DRIVE implementation, the peer-incentivization baselines used in the paper, four social-dilemma environments, training utilities, and scripts for reproducing the empirical and theoretical figures.

DRIVE augments independent multi-agent reinforcement learning with a local request-response protocol:

DRIVE does not learn an incentive function, require a fixed token magnitude, or extend the agents' action spaces.

In the two-agent Prisoner's Dilemma, the protocol reverses the unilateral temptation and sucker incentives in the steady exploitation regime. This removes greed and fear and makes cooperation individually rational under the assumptions developed in the paper.

Reward-surface interpretation

The continuous mean-field analysis compares the unshaped Prisoner's Dilemma reward landscape (left) with the DRIVE-shaped landscape (right). The unshaped gradient field points toward defection, whereas DRIVE redirects local incentives toward mutual cooperation.

Robustness to stochastic reward perturbations

For independently perturbed rewards separated by a gap (D), the ordering probability is

$$\Pr[\tilde r > \tilde r'] = \Phi!\left(\frac{D}{\sqrt{2}\sigma}\right).$$

The paper distinguishes between per-agent perturbations, which preserve each agent's local payoff ordering, and independent payoff-entry perturbations, which may alter adjacent Prisoner's Dilemma inequalities.

The evaluation includes MATE, MEDIATE, LToS, LIO, IA, and naive independent learning across stationary rewards and five reward-change settings.

Reward-change functions

The first four non-stationary transformations are shared positive affine reward transformations. The noisy setting evaluates robustness beyond that exact invariance class.

The plotting code also supports DRIVE ablations such as DRIVE-ungated, DRIVE-instant, and DRIVE-retrust when corresponding runs are available.

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The experiments in the paper were run on CPUs. A CUDA-capable GPU is therefore not required.

The training entry point is:

python -m train ALGORITHM DOMAIN [OPTIONS]

Arguments

Examples

Train DRIVE in the stationary IPD:

python -m train DRIVE Matrix-IPD

Train DRIVE in Coin-4 with damped-cosine reward changes and seed 7:

python -m train DRIVE CoinGame-4 -d cos_damped -s 7

Train MEDIATE in Harvest-12 under noisy rewards for 1,000 epochs:

python -m train MEDIATE Harvest-12 -d noisy -s 1 -e 1000

Write outputs to another directory:

python -m train DRIVE CoinGame-2 -d linear -o output_linear

Each run creates a timestamped subdirectory below the selected output folder and stores the training statistics used by the plotting utilities. Global parameters such as learning rate, discount factor, number of episodes per epoch, and default epoch count are defined in train/settings.py. Algorithm-specific defaults are implemented in src/controllers/.

Reproducing a sweep

A typical shell sweep is:

for seed in $(seq 1 20); do
  for env in Matrix-IPD CoinGame-2 CoinGame-4 Harvest-12; do
    for alg in DRIVE MATE MEDIATE LToS LIO IA IAC; do
      for drift in identity linear exponential_decay stepwise_increase cos_damped noisy; do
        python -m train "$alg" "$env" -d "$drift" -s "$seed"
      done
    done
  done
done

Adjust the sweep to the available compute budget. The main paper reports 20 runs over 4,000 epochs. To reproduce the results from the paper, use:

./run.sh

The plotting entry point is:

python -m plot MODE [OPTIONS]

Common options

When multiple domains are supplied, --metrics must contain the same number of entries and is matched positionally.

Main benchmark plots

Generate the primary metric for all four environments and all reward settings:

python -m plot benchmark --static-baselines

Generate selected environment-metric pairs:

python -m plot benchmark \
  -e Matrix-IPD CoinGame-2 CoinGame-4 Harvest-12 \
  -m cooperation_rate own_coin_prob own_coin_prob sustainability \
  --static-baselines

Restrict the plot to selected algorithms and reward settings:

python -m plot benchmark \
  -a DRIVE MATE MEDIATE LToS \
  -d identity linear noisy \
  -e Matrix-IPD \
  -m cooperation_rate

Generated benchmark figures are written to plots/benchmark/

Additional metrics

Peace in Harvest-12:

python -m plot benchmark -e Harvest-12 -m peace

Efficiency/social welfare across all domains:

python -m plot benchmark \
  -e Matrix-IPD CoinGame-2 CoinGame-4 Harvest-12 \
  -m undiscounted_returns undiscounted_returns undiscounted_returns undiscounted_returns

Equality across all domains:

python -m plot benchmark \
  -e Matrix-IPD CoinGame-2 CoinGame-4 Harvest-12 \
  -m equality equality equality equality

Drift-function plots

python -m plot drifts

To draw only selected functions:

python -m plot drifts -d identity linear noisy

Introductory comparison

Generate the motivating comparison from available runs:

python -m plot intro \
  --dir output_intro \
  -d identity scale_up \
  -e Matrix-IPD \
  -m cooperation_rate \
  -a DRIVE MATE IA

The intro mode aggregates the final portion of the corresponding runs into comparison bars.

DRIVE ablations

python -m plot ablations \
  -e Matrix-IPD \
  -m cooperation_rate

This mode selects the available DRIVE-* ablation labels and uses stationary rewards.

Theoretical noise plots

Generate both the payoff-density overlap and ordering-probability views:

python -m plot theory -d noise -m both

Generate only one component:

python -m plot theory -d noise -m overlap
python -m plot theory -d noise -m ordering

Mean-field reward surfaces and vector fields

Generate both the unshaped and DRIVE-shaped theoretical plots:

python -m plot theory -a IAC DRIVE

Generate only one case:

python -m plot theory -a IAC
python -m plot theory -a DRIVE

The theoretical figures are written below:

plots/theory/

@article{altmann2026drive,
  title   = {Dynamic Reward Incentives for Emergent Cooperation under Changing Rewards},
  author  = {Altmann, Philipp and Zorn, Maximilian and Koenig, Sven and Phan, Thomy},
  journal = {Transactions on Machine Learning Research},
  year    = {2026},
  url     = {https://openreview.net/forum?id=9Ltu1HV2YI}
}