Running parameter sweeps
In the previous example, we built a four-round market with a change in private information halfway through. We're now going to run that experiment with different reasoning efforts, then add another seed to make a small parameter sweep.
Create an experiment.yaml file
Download the complete files and extract them into your experiment folder, keeping the examples/ directory structure.
Create or edit this stage's files in examples/tutorials/parameter_sweeps.
We'll add reasoning_effort and seed to variables, so we can change them from Python without rewriting the YAML file.
Both agent groups use ${variables.reasoning_effort}, and the experiment uses ${variables.seed}.
Create a new file called experiment.yaml in examples/tutorials/parameter_sweeps, and copy the contents of the snippet below into it.
The highlighted lines show what has changed since the previous example.
variables:
buyer_values: ['100', '80', '60']
seller_costs: ['40', '60', '90']
shifted_buyer_values: ['120', '100', '80']
shifted_seller_costs: ['60', '80', '110']
shift_period: 3
buyer_endowment:
cash: '150'
units:
asset: 0
seller_endowment:
cash: '0'
units:
asset: 1
reasoning_effort: none
seed: 42
experiment:
name: parameter_sweeps
seed: ${variables.seed}
output_dir: results
protocol:
period_duration_seconds: 30
periods: 4
hooks:
period_start:
- type: examples.tutorials.parameter_sweeps.hooks.ResetEndowments
params:
buyer_endowment: ${variables.buyer_endowment}
seller_endowment: ${variables.seller_endowment}
assets:
- id: asset
name: Good
markets:
- id: asset
price_min: '0'
price_max: '150'
agents:
- id: buyer
n: 3
lm:
provider: openai
model: gpt-5.4-mini
temperature: null
reasoning_effort: ${variables.reasoning_effort}
max_tokens: 4096
prompt:
instructions:
content: |
You are a buyer. You can trade one unit per round.
Your private value appears in each market update.
Your earnings are your value minus the purchase price.
If you do not trade, your earnings are zero.
Maximise your earnings. Do not buy above your value. Do not sell.
context_providers:
- type: examples.tutorials.parameter_sweeps.providers.PrivateValueProvider
params:
buyer_values: ${variables.buyer_values}
seller_costs: ${variables.seller_costs}
shifted_buyer_values: ${variables.shifted_buyer_values}
shifted_seller_costs: ${variables.shifted_seller_costs}
shift_period: ${variables.shift_period}
- type: snapshot_period_timing
- type: snapshot_order_book
- type: snapshot_own_orders
- type: snapshot_own_holdings
- type: snapshot_last_action_outcome
initial_holdings: ${variables.buyer_endowment}
trading_constraints:
allowed_sides: [bid]
max_units:
asset: 1
max_order_quantity: 1
max_outstanding_orders: 1
- id: seller
n: 3
lm:
provider: openai
model: gpt-5.4-mini
temperature: null
reasoning_effort: ${variables.reasoning_effort}
max_tokens: 4096
prompt:
instructions:
content: |
You are a seller. You can trade one unit per round.
Your private cost appears in each market update.
Your earnings are your sale price minus your cost.
If you do not trade, your earnings are zero.
Maximise your earnings. Do not sell below your cost. Do not buy.
context_providers:
- type: examples.tutorials.parameter_sweeps.providers.PrivateValueProvider
params:
buyer_values: ${variables.buyer_values}
seller_costs: ${variables.seller_costs}
shifted_buyer_values: ${variables.shifted_buyer_values}
shifted_seller_costs: ${variables.shifted_seller_costs}
shift_period: ${variables.shift_period}
- type: snapshot_period_timing
- type: snapshot_order_book
- type: snapshot_own_orders
- type: snapshot_own_holdings
- type: snapshot_last_action_outcome
initial_holdings: ${variables.seller_endowment}
trading_constraints:
allowed_sides: [ask]
max_order_quantity: 1
max_outstanding_orders: 1
The default values are still none and 42.
Each call to aigora.run() can replace these with the values we supply through its variables argument.
Reuse the provider and hook
The private information and account resets work exactly as before.
Create providers.py and hooks.py alongside experiment.yaml, using the same code from the previous example:
import aigora
class PrivateValueProvider:
def __init__(
self,
buyer_values,
seller_costs,
shifted_buyer_values,
shifted_seller_costs,
shift_period,
):
self.shift_period = shift_period
self.values = {}
for number, values in enumerate(
zip(buyer_values, shifted_buyer_values), start=1
):
self.values[f"buyer_{number}"] = values
for number, costs in enumerate(
zip(seller_costs, shifted_seller_costs), start=1
):
self.values[f"seller_{number}"] = costs
def get_context(self, context: aigora.ProviderContext) -> str:
agent_id = context.agent.agent_id
original, shifted = self.values[agent_id]
value = shifted if context.experiment.period >= self.shift_period else original
label = "value" if agent_id.startswith("buyer_") else "cost"
return f"Your private {label} for this round is {value}."
import aigora
class ResetEndowments:
def __init__(self, buyer_endowment, seller_endowment):
self.buyer = buyer_endowment
self.seller = seller_endowment
def on_period_start(self, context: aigora.PeriodStartContext) -> None:
for agent_id, agent in context.agents.items():
endowment = self.buyer if agent_id.startswith("buyer_") else self.seller
agent.holdings = aigora.Holdings(**endowment)
Run the experiment
This time, we'll put our call to aigora.run() inside a loop.
The product function pairs each reasoning effort with each seed, and the loop runs one experiment for each pair.
Create run.py alongside the other files:
import asyncio
from itertools import product
import aigora
async def main():
for effort, seed in product(("none", "medium"), (42,)):
run_dir = await aigora.run(
"examples/tutorials/parameter_sweeps/experiment.yaml",
variables={"reasoning_effort": effort, "seed": seed},
display=False,
)
print(f"{effort}, seed {seed}: Experiment results saved to {run_dir}")
if __name__ == "__main__":
asyncio.run(main())
asyncio.run(main()) starts the loop, and await lets each experiment finish before the next one begins.
Setting display=False hides the live dashboard, leaving a printed results path for each run.
Run the script from your experiment folder (the folder containing examples/):
This starts two four-round experiments, both with seed 42: one with reasoning effort none, and one with medium.
Each has its own results directory, containing trades.csv, holdings.csv, and traces/.
See Output files for the full descriptions.
Add another parameter
To include a second seed, replace (42,) with (42, 43) in the product call:
The same loop now runs four experiments: none with seeds 42 and 43, followed by medium with seeds 42 and 43.
The seed controls local randomness; model responses and the order of agent actions can still vary between runs.
You can use this pattern for any setting you declare in variables.
For example, add a variable for round duration, use it in the YAML field, and pass its value in the loop.
See Variables and overrides for more detail.
Download the complete files if you want a separate copy of this stage.