bnelearn.strategy module

Implementations of strategies for playing in Auctions and Matrix Games.

class bnelearn.strategy.ClosureStrategy(closure: Callable, parallel: int = 0, mute=False)

Bases: Strategy

A strategy specified by a closure

Args:

closure: Callable a function or lambda that defines the strategy parallel: int (optional) maximum number of processes for parallel execution of closure. Default is

0/1 (i.e. no parallelism)

play(inputs, deterministic: bool = False)

Takes (private) information as input and decides on the actions an agent should play.

to(device: str)

Compatibility to Pytorch’s .to().

class bnelearn.strategy.FictitiousNeuralPlayStrategy(n_actions, beliefs, init_weight_normalization=False)

Bases: MatrixGameStrategy, Module

An implementation of the concept of Fictitious Play with NN. An implementation inspired by: https://www.groundai.com/project/deep-fictitious-play-for-stochastic-differential-games2589/2 Take the beliefs about others strategies as input for the NN.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class bnelearn.strategy.FictitiousPlayMixedStrategy(game: Game, initial_beliefs: Optional[Iterable[Tensor]] = None)

Bases: FictitiousPlaySmoothStrategy

Play (communicate) probabilities for play (same as in smooth FP) instead of one action. One strategy should be shared among all players such that they share the same beliefs. This is purely fictitious since it does not simulate actions.

play(player_position) → Tensor

Takes (private) information as input and decides on the actions an agent should play.

update_observations(actions: None)

class bnelearn.strategy.FictitiousPlaySmoothStrategy(game: Game, initial_beliefs: Optional[Iterable[Tensor]] = None)

Bases: FictitiousPlayStrategy

Implementation based on Fudenberg (1999) but extended by smooth fictitious play. Randomize action by taking the softmax over the expected utilities for each action and sample. Also, add a temperature (tau) that ensures convergence by becoming smaller.

play(player_position) → Tensor

Takes (private) information as input and decides on the actions an agent should play.

update_tau(param=0.9)

Updates temperature parameter

class bnelearn.strategy.FictitiousPlayStrategy(game: MatrixGame, initial_beliefs: Optional[Iterable[Tensor]] = None)

Bases: Strategy

Based on description in: Fudenberg, 1999 - The Theory of Learning, Chapter 2.2 Always play best response (that maximizes utility based on current beliefs).

play(player_position: int)

Takes (private) information as input and decides on the actions an agent should play.

update_beliefs()

Update beliefs about play

update_observations(actions: Iterable[Tensor])

class bnelearn.strategy.MatrixGameStrategy(n_actions, init_weights=None, init_weight_normalization=False)

Bases: Strategy, Module

A dummy neural network that encodes and returns a mixed strategy

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

play(inputs=None, batch_size=1)

Takes (private) information as input and decides on the actions an agent should play.

to(device)

Moves and/or casts the parameters and buffers.

This can be called as

to(device=None, dtype=None, non_blocking=False)

to(dtype, non_blocking=False)

to(tensor, non_blocking=False)

to(memory_format=torch.channels_last)

Its signature is similar to torch.Tensor.to(), but only accepts floating point or complex dtypes. In addition, this method will only cast the floating point or complex parameters and buffers to dtype (if given). The integral parameters and buffers will be moved device, if that is given, but with dtypes unchanged. When non_blocking is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.

See below for examples.

Note

This method modifies the module in-place.

Args:

device (torch.device): the desired device of the parameters

and buffers in this module

dtype (torch.dtype): the desired floating point or complex dtype of

the parameters and buffers in this module

tensor (torch.Tensor): Tensor whose dtype and device are the desired

dtype and device for all parameters and buffers in this module

memory_format (torch.memory_format): the desired memory

format for 4D parameters and buffers in this module (keyword only argument)

Returns:

Module: self

Examples:

>>> # xdoctest: +IGNORE_WANT("non-deterministic")
>>> linear = nn.Linear(2, 2)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
        [-0.5113, -0.2325]])
>>> linear.to(torch.double)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
        [-0.5113, -0.2325]], dtype=torch.float64)
>>> # xdoctest: +REQUIRES(env:TORCH_DOCTEST_CUDA1)
>>> gpu1 = torch.device("cuda:1")
>>> linear.to(gpu1, dtype=torch.half, non_blocking=True)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
        [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1')
>>> cpu = torch.device("cpu")
>>> linear.to(cpu)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
        [-0.5112, -0.2324]], dtype=torch.float16)

>>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble)
>>> linear.weight
Parameter containing:
tensor([[ 0.3741+0.j,  0.2382+0.j],
        [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128)
>>> linear(torch.ones(3, 2, dtype=torch.cdouble))
tensor([[0.6122+0.j, 0.1150+0.j],
        [0.6122+0.j, 0.1150+0.j],
        [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)

training: bool

class bnelearn.strategy.NeuralNetStrategy(input_length: int, hidden_nodes: Iterable[int], hidden_activations: Iterable[Module], ensure_positive_output: Optional[Tensor] = None, output_length: int = 1, dropout: float = 0.0, mixed_strategy: Optional[str] = None, bias: bool = True, res_net: bool = False)

Bases: Strategy, Module

A strategy played by a fully connected neural network

Args:

input_length:

dimension of the input layer

hidden_nodes:

Iterable of number of nodes in hidden layers

hidden_activations:

Iterable of activation functions to be used in the hidden layers. Should be instances of classes defined in torch.nn.modules.activation

ensure_positive_output (optional): torch.Tensor

When provided, will check whether the initialized model will return a positive bid anywhere at the given input tensor. Otherwise, the weights will be reinitialized.

output_length (optional): int

length of output/action vector defaults to 1 (currently given last for backwards-compatibility)

dropout (optional): float

If not 0, applies AlphaDropout (https://pytorch.org/docs/stable/nn.html#torch.nn.AlphaDropout) to dropout share of nodes in each hidden layer during training.

mixed_strategy (otional): str

Which distribution to use when the strategy should be mixed.

res_net (optional): bool

Switch to use ResNet skip connections, s.t. only the difference from a ruthful strategy is learned.

forward(x, deterministic=False, pretrain=False)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_gradient_norm()

Get the norm of the gradient

classmethod load(path: str, device='cpu')

Initializes a saved NeuralNetStrategy from path.

play(inputs, deterministic: bool = False)

Takes (private) information as input and decides on the actions an agent should play.

pretrain(input_tensor: Tensor, iterations: int, transformation: Optional[Callable] = None)

Performs iters steps of supervised learning on input tensor, in order to find an initial bid function that is suitable for learning.

args:

input: torch.Tensor, same dimension as self.input_length iters: number of iterations for supervised learning transformation (optional): Callable. Defaulting to identity function if input_length == output_length

returns: Nothing

reset(ensure_positive_output=None)

Re-initialize weights of the Neural Net, ensuring positive model output for a given input.

training: bool

class bnelearn.strategy.Strategy

Bases: ABC

A Strategy to map (optional) private inputs of a player to actions in a game.

abstract play(inputs)

Takes (private) information as input and decides on the actions an agent should play.

pretrain(input_tensor, iterations, transformation=None)

If implemented by subclass, pretrains the strategy to yield desired initial outputs.

class bnelearn.strategy.TruthfulStrategy

Bases: Strategy, Module

A strategy that plays truthful valuations.

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

play(inputs, deterministic: bool = False)

Takes (private) information as input and decides on the actions an agent should play.

training: bool