bnelearn.experiment.configuration_manager module

class bnelearn.experiment.configuration_manager.ConfigurationManager(experiment_type: str, n_runs: int, n_epochs: int, seeds: Optional[Iterable[int]] = None)

Bases: object

The class provides a ‘front-end’ for the whole package. It allows for creation of a full and consistent ExperimentConfiguration, as defined by the ExperimentConfig dataclass. It manages all the defaults, including those specific for each experiment type, auto-inits the parameters that are not supposed to be initialized manually, and allows to selectively change any part of the configuration, while also performing a parameter and consistency check before creating the final configuration object.

The workflow with the class API is as follows:

1. Init class object with the experiment type string, n_runs and n_epochs. For possible experiment types see ConfigurationManager.experiment_types

   1. __init__ calls get_default_config_members method to get default configuration members.

   2. Based on the experiment type, __init__ calls the appropriate ancillary _init_experiment_type. It sets the default parameters specific for the given experiment type.

2. (Optional step) Call set_config_member methods (e.g. set_setting) in a chain style, each methods allows to selectively set any parameter of a corresponding config member to a new arbitrary value, while leaving all the parameters not specified by the user intact - with their default values.

3. Call the get_config method to get a ready configuration object and an experiment class corresponding to the experiment type (the latter needed for an easy instantiation of the Experiment)

   1. get_config calls _post_init, which inits the parameters which shouldn’t be set manually, checks for consistency between the related parameters and validates whether each parameter is in an appropriate value range. Then, it calls the type specific _post_init_experiment_type method which performs all the same things, but specific for the experiment type.

   2. get_config creates and returns the final and valid configuration object alongside the experiment class.

Example of class usage:

experiment_config, experiment_class = \
    ConfigurationManager(
        experiment_type='multiunit', n_runs=1, n_epochs=20
    ) \
    .set_logging(log_root_dir=log_root_dir) \
    .set_setting(payment_rule='discriminatory') \
    .set_learning(model_sharing=False) \
    .set_hardware() \
    .get_config()

experiment_class(experiment_config).run()

static compare_two_experiment_configs(conf1: ExperimentConfig, conf2: ExperimentConfig) → bool

Checks whether two given configurations are identical (deep comparison)

static experiment_config_could_be_saved_properly(exp_config: ExperimentConfig) → bool

Tests whether the given config could be serialized and deserialized properly.

experiment_types = {'affiliated_observations': (<class 'bnelearn.experiment.single_item_experiment.AffiliatedObservationsExperiment'>, <function ConfigurationManager._init_affiliated_observations>, <function ConfigurationManager._post_init_affiliated_observations>), 'all_pay_uniform_symmetric': (<class 'bnelearn.experiment.single_item_experiment.UniformSymmetricPriorSingleItemExperiment'>, <function ConfigurationManager._init_single_item_uniform_symmetric>, <function ConfigurationManager._post_init_single_item_uniform_symmetric>), 'crowdsourcing': (<class 'bnelearn.experiment.single_item_experiment.ContestExperiment'>, <function ConfigurationManager._init_crowdsourcing>, <function ConfigurationManager._post_init_crowdsourcing>), 'llg': (<class 'bnelearn.experiment.combinatorial_experiment.LLGExperiment'>, <function ConfigurationManager._init_llg>, <function ConfigurationManager._post_init_llg>), 'llg_full': (<class 'bnelearn.experiment.combinatorial_experiment.LLGFullExperiment'>, <function ConfigurationManager._init_llg_full>, <function ConfigurationManager._post_init_llg_full>), 'llllgg': (<class 'bnelearn.experiment.combinatorial_experiment.LLLLGGExperiment'>, <function ConfigurationManager._init_llllgg>, <function ConfigurationManager._post_init_llllgg>), 'llllrrg': (<class 'bnelearn.experiment.combinatorial_experiment.LLLLRRGExperiment'>, <function ConfigurationManager._init_llllrrg>, <function ConfigurationManager._post_init_llllrrg>), 'mineral_rights': (<class 'bnelearn.experiment.single_item_experiment.MineralRightsExperiment'>, <function ConfigurationManager._init_mineral_rights>, <function ConfigurationManager._post_init_mineral_rights>), 'multiunit': (<class 'bnelearn.experiment.multi_unit_experiment.MultiUnitExperiment'>, <function ConfigurationManager._init_multiunit>, <function ConfigurationManager._post_init_multiunit>), 'single_item_asymmetric_beta': (<class 'bnelearn.experiment.single_item_experiment.TwoPlayerAsymmetricBetaPriorSingleItemExperiment'>, <function ConfigurationManager._init_single_item_asymmetric_beta>, <function ConfigurationManager._post_init_single_item_asymmetric_beta>), 'single_item_asymmetric_uniform_disjunct': (<class 'bnelearn.experiment.single_item_experiment.TwoPlayerAsymmetricUniformPriorSingleItemExperiment'>, <function ConfigurationManager._init_single_item_asymmetric_uniform_disjunct>, <function ConfigurationManager._post_init_single_item_asymmetric_uniform_disjunct>), 'single_item_asymmetric_uniform_overlapping': (<class 'bnelearn.experiment.single_item_experiment.TwoPlayerAsymmetricUniformPriorSingleItemExperiment'>, <function ConfigurationManager._init_single_item_asymmetric_uniform_overlapping>, <function ConfigurationManager._post_init_single_item_asymmetric_uniform_overlapping>), 'single_item_gaussian_symmetric': (<class 'bnelearn.experiment.single_item_experiment.GaussianSymmetricPriorSingleItemExperiment'>, <function ConfigurationManager._init_single_item_gaussian_symmetric>, <function ConfigurationManager._post_init_single_item_gaussian_symmetric>), 'single_item_uniform_symmetric': (<class 'bnelearn.experiment.single_item_experiment.UniformSymmetricPriorSingleItemExperiment'>, <function ConfigurationManager._init_single_item_uniform_symmetric>, <function ConfigurationManager._post_init_single_item_uniform_symmetric>), 'splitaward': (<class 'bnelearn.experiment.multi_unit_experiment.SplitAwardExperiment'>, <function ConfigurationManager._init_splitaward>, <function ConfigurationManager._post_init_splitaward>), 'tullock_contest': (<class 'bnelearn.experiment.single_item_experiment.ContestExperiment'>, <function ConfigurationManager._init_tullock>, <function ConfigurationManager._post_init_tullock>)}

static get_class_by_experiment_type(experiment_type: str)

Given an experiment type, returns the corresponding experiment class which could be initialized

get_config()

Performs the _post_init, creates and returns the final ExperimentConfig object alongside with the appropriate experiment class

static get_default_config_members() → Tuple[RunningConfig, SettingConfig, LearningConfig, LoggingConfig, HardwareConfig]

Creates with default (or most common) parameters and returns members of the ExperimentConfig

static load_experiment_config(experiment_log_dir=None)

Retrieves stored configurations from JSON and turns them into ExperimentConfiguration object By default creates configuration from the file stored alongside the running script

Parameters:

experiment_log_dir – full path except for the file name, current working directory by default

Returns:

ExperimentConfiguration object

set_hardware(cuda: bool = 'None', specific_gpu: int = 'None', fallback: bool = 'None', max_cpu_threads: int = 'None')

Sets only the parameters of hardware which were passed, returns self

set_learning(model_sharing: bool = 'None', learner_type: str = 'None', learner_hyperparams: dict = 'None', optimizer_type: str = 'None', optimizer_hyperparams: dict = 'None', scheduler_type: str = 'None', scheduler_hyperparams: dict = 'None', hidden_nodes: List[int] = 'None', pretrain_iters: int = 'None', smoothing_temperature: bool = 'None', batch_size: int = 'None', hidden_activations: List[Module] = 'None', redraw_every_iteration: bool = 'None', mixed_strategy: str = 'None', pretrain_to_bne: int = 'None', value_contest: bool = True)

Sets only the parameters of learning which were passed, returns self

set_logging(enable_logging: bool = 'None', log_root_dir: str = 'None', util_loss_batch_size: int = 'None', util_loss_opponent_batch_size: int = 'None', util_loss_grid_size: int = 'None', eval_frequency: int = 'None', eval_batch_size: int = 'None', plot_frequency: int = 'None', plot_points: int = 'None', plot_show_inline: bool = 'None', log_metrics: dict = 'None', best_response: bool = 'None', save_tb_events_to_csv_aggregate: bool = 'None', save_tb_events_to_csv_detailed: bool = 'None', save_tb_events_to_binary_detailed: bool = 'None', save_models: bool = 'None', save_figure_to_disk_png: bool = 'None', save_figure_to_disk_svg: bool = 'None', save_figure_data_to_disk: bool = 'None', cache_eval_actions: bool = 'None', export_step_wise_linear_bid_function_size: bool = 'None', experiment_dir: str = 'None', experiment_name: str = 'None')

Sets only the parameters of logging which were passed, returns self

set_setting(n_players: int = 'None', payment_rule: str = 'None', risk: float = 'None', n_items: int = 'None', common_prior: Distribution = 'None', valuation_mean: float = 'None', valuation_std: float = 'None', u_lo: list = 'None', u_hi: list = 'None', gamma: float = 'None', correlation_types: str = 'None', correlation_groups: List[List[int]] = 'None', correlation_coefficients: List[float] = 'None', pretrain_transform: callable = 'None', constant_marginal_values: bool = 'None', item_interest_limit: int = 'None', efficiency_parameter: float = 'None', core_solver: str = 'None', tullock_impact_factor: float = 'None', impact_function: str = 'None', crowdsourcing_values: Optional[List] = None)

Sets only the parameters of setting which were passed, returns self. Using None here and below Args:

n_players: The number of players in the game. payment_rule: The payment rule to be used. risk: A strictly positive risk-parameter. A value of 1 corresponds to risk-neutral agents,

values <1 indicate risk-aversion.

common_prior: The common type distribution shared by all players, explicitly given as a

torch.distributions.Distribution object. Gaussian distribution.

valuation_mean: The expectation of the valuation distribution, when implicitly setting up a

Gaussian distribution.

valuation_std: The standard deviation of the valuation distribution, when implicitly setting up a

Gaussian distribution.

u_lo: Lower bound of valuation distribution, when implicitly setting up a Uniform distribution. u_hi: Upper bound of valuation distribution, when implicitly setting up a Uniform distribution. gamma: Correlation parameter for correlated value distributions of bidders. (Relevant Settings: LLG) correlation_types: Specifies the type of correlation model. (Most relevant settings: LLG) correlation_groups: A list of lists that ‘groups’ players into correlated subsets. All players

should be part of exactly one sublist. (Relevant settings: LLG)

correlation_coefficients: List of correlation coefficients for each

group specified with correlation_groups.

n_items: Number of items to sell. pretrain_transform: A function used to explicitly give the desired behavior in pretraining for

given neural net inputs. Defaults to identity, i.e. truthful bidding.

constant_marginal_values: Whether or not the bidders have constant marginal values in

multi-unit auctions.

item_interest_limit: Whether or not the bidders have a lower demand in units than available in

multi-unit auctions.

efficiency_parameters: Efficiency parameter in split-award auction. core_solver: Specifies which solver should be used to calculate core prices.

Should be one of ‘NoCore’, ‘mpc’, ‘gurobi’, ‘cvxpy’ (Relevant settings: LLLLGG)

Returns:

self with updated parameters.