Experiments¶
-
class passengersim.experiments.Experiments(config: Config, output_dir: Path | None | False =
None, *, pickle: bool | str =False, html: bool | str ='passengersim_output', hide_from_git: bool =True)[source]¶ Bases:
CallbackMixinManage and run a collection of PassengerSim experiments.
This class stores a base
Config, creates per-experiment configurations, orchestrates sequential or parallel execution, and collects the results into aContrastdictionary. It also manages output paths, report generation, and optional result caching.Initialize the experiment manager.
- Parameters:
- config : Config¶
Base configuration used to create per-experiment deep copies.
- output_dir : pathlib.Path or None or False, optional¶
Directory where all experiment outputs are written. If
None(default), outputs are placed in tag-named subdirectories of the current directory. IfFalse, disable automatic file exports, simulation-state snapshots, raw files, and the combined HTML report, even when requested by the config or an experiment function. Database results use OS temporary storage instead of configured destinations; internal temporary files may still be created.- pickle : bool or str, default False¶
If truthy, ensure the base config is set up to write pickle output. If
True, the default filename stem"passengersim_output"is used.- html : bool or str, default "passengersim_output"¶
If truthy, ensure the base config is set up to write HTML output. If
True, the default filename stem"passengersim_output"is used.- hide_from_git : bool, default True¶
If True and
output_diris set, write a.gitignorefile that prevents git from tracking generated outputs.
- Returns:
None
Methods
__init__(config[, output_dir, pickle, html, ...])Initialize the experiment manager.
Add callback events to the simulation event queue.
Attach this object's manual callbacks to another simulation.
begin_sample_callback([callback, priority, ...])Register a function to be triggered at the beginning of each sample.
Get all manually attached callback functions.
daily_callback([callback, priority, ...])Register a function to be triggered each day during a sample.
end_sample_callback([callback, priority, ...])Register a function to be triggered at the end of each sample.
existing([external])Create an experiment that uses existing results, rather than running a simulation.
finalize_sample_callback([callback, ...])Register a synchronous callback after sample bookkeeping finishes.
initialize_sample_callback([callback, ...])Register a synchronous callback before sample demand generation.
remove_callback(callback, *[, missing_ok])Detach manually attached callbacks or tracers.
run([use_existing, tag, check_versions, ...])Run the experiments, choosing sequential or parallel execution automatically.
Adopt all globally registered callbacks (deprecated).
validate()Validate all experiment tags and callables against the base config.
Attributes
Every callback attached to the
begin_samplehook, in the order they run.Every callback attached to the
dailyhook, in the order they run.Every callback attached to the
end_samplehook, in the order they run.Every callback attached to the
finalize_samplehook, in the order they run.Every callback attached to the
initialize_samplehook, in the order they run.Path of the HTML report written after the most recent run.
Retained simulation objects from the most recent run.
-
__init__(config: Config, output_dir: Path | None | False =
None, *, pickle: bool | str =False, html: bool | str ='passengersim_output', hide_from_git: bool =True)[source]¶ Initialize the experiment manager.
- Parameters:
- config : Config¶
Base configuration used to create per-experiment deep copies.
- output_dir : pathlib.Path or None or False, optional¶
Directory where all experiment outputs are written. If
None(default), outputs are placed in tag-named subdirectories of the current directory. IfFalse, disable automatic file exports, simulation-state snapshots, raw files, and the combined HTML report, even when requested by the config or an experiment function. Database results use OS temporary storage instead of configured destinations; internal temporary files may still be created.- pickle : bool or str, default False¶
If truthy, ensure the base config is set up to write pickle output. If
True, the default filename stem"passengersim_output"is used.- html : bool or str, default "passengersim_output"¶
If truthy, ensure the base config is set up to write HTML output. If
True, the default filename stem"passengersim_output"is used.- hide_from_git : bool, default True¶
If True and
output_diris set, write a.gitignorefile that prevents git from tracking generated outputs.
- Returns:
None
- property sims : dict[str, Simulation | MultiSimulation]¶
Retained simulation objects from the most recent run.
- Returns:
dict[str, Simulation or MultiSimulation] – Mapping from experiment tag to the simulation instance that produced the corresponding results.
- Raises:
ValueError – If
retain_sims=Truewas not passed torun().
-
existing(external: GenericSimulationTables | str | bytes | PathLike | None =
None) Experiment[source]¶ Create an experiment that uses existing results, rather than running a simulation.
- Parameters:
- external : GenericSimulationTables or path-like, optional¶
If provided, this should be an existing SimulationTables result or a path to an existing output file containing the results for this experiment. If this is provided, the experiment will skip running the simulation and instead load the results from the given file. This is useful for cases where the simulation has already been run and the results are saved, but you want to include those results in a report with other experiments, without re-running the simulation. If given a path but the given file does not exist or cannot be loaded, an error will be raised.
- Returns:
Experiment – An experiment that will use the given existing results when run, rather than running a simulation.
- add_callback_events()¶
Add callback events to the simulation event queue.
All callbacks due at the same timestamp are placed in one event. The native event queue orders events by time alone, so separate events at one timestamp would run in an arbitrary order. Within one event, they run in this order:
RM systems;
then the hooks in lifecycle order, so that at the first DCP every
begin_samplecallback runs before anydailyone, and at the final DCP everydailycallback runs before anyend_sampleone;within each hook, configured callbacks before manual ones, each in registration (or declaration) order.
RM systems stay first so that a priority of -1, the RM systems’ own, still runs a callback after them, as documented for each hook.
- apply_callback_functions(sim: CallbackMixin) None¶
Attach this object’s manual callbacks to another simulation.
- Parameters:
- sim : CallbackMixin¶
The simulation (or experiments) to receive the callbacks. Tracers are copied fresh; other callables are shared.
-
begin_sample_callback(callback: Callable[[CallbackMixin], dict | None] | GenericTracer | None =
None, *, priority: int | None =None, skip_burn: bool | None =None, name: str | None =None) Callable[[CallbackMixin], dict | None] | GenericTracer¶ Register a function to be triggered at the beginning of each sample.
The callback function will be triggered after initial setup including all RM steps for the initial DCP, but before any customers can arrive.
It runs at an offset, in seconds, from the trigger time of the first DCP. The offset is the priority option if given, else the callback’s own
priorityattribute if it has one, and otherwise one second, which is after the first DCP’s RM systems (-1) and bookkeeping (0). A negativepriorityruns it before that bookkeeping, or, below -1, before the first DCP’s RM steps as well. No customer arrives within five seconds of a DCP, so apriorityfrom 1 to 4 always runs before the first arrival; a larger one may not. Apriorityof zero is not allowed, as for a daily callback.- Parameters:
- callback : Callable[[Simulation], dict | None] or GenericTracer, optional¶
The callback function to register. It should accept a single argument, which will be the Simulation object, and return a dictionary of interesting things to store, or nothing. Omit it to use this method as a decorator with options.
- priority : int, optional¶
Offset in seconds from the first DCP’s trigger time; see above.
- skip_burn : bool, optional¶
If True, the callback is not called during burn-in samples.
- name : str, optional¶
Store returned dictionaries under
callback_data.<name>instead of the shared hook key; for a tracer, its result name.
- Returns:
Callable or GenericTracer – The callback, or the fresh copy of a tracer. Without callback, a decorator that registers the function it decorates.
- property begin_sample_callbacks : tuple[Callable, ...]¶
Every callback attached to the
begin_samplehook, in the order they run.Includes RM systems and configured callbacks. This is a read-only snapshot: use the
begin_sample_callbackmethod to attach a callback.
- callback_functions() dict[str, list[Callable]]¶
Get all manually attached callback functions.
Callbacks constructed from
config.callbacksdeclarations are excluded. They are owned by this object’s config and are rebuilt by any simulation that runs that config, so copying them elsewhere (e.g. viaapply_callback_functions()or worker forwarding) would attach them twice. Carrier RM systems are excluded for the same reason: every simulation builds its own.- Returns:
dict[str, list[Callable]] – Mapping of
"<hook>_callbacks"to the manual callbacks for each hook, in registration order. A tracer is listed as itself under each hook it runs on.
-
daily_callback(callback: collections.abc.Callable[[CallbackMixin, int], dict | None] | GenericTracer | RmSys | None =
None, *, priority: int | None =None, skip_burn: bool | None =None, name: str | None =None) collections.abc.Callable[[CallbackMixin, int], dict | None] | GenericTracer¶ Register a function to be triggered each day during a sample.
The callback runs at an offset, in seconds, from the overnight trigger time. The offset is the priority option if given, else the callback’s own
priorityattribute if it has one, and otherwise one second. RM systems run at -1 and the simulation’s own DCP and daily data capture at 0, so by default a callback sees each day after both have finished. Use a negativepriorityto run it before the simulation’s bookkeeping, or below -1 to run it before the RM systems as well.A
priorityof zero is not allowed, because it would tie with the simulation’s own bookkeeping in an arbitrary order. It is rejected here, and again when a run starts if it was set after registration. Theprioritymust also be a whole number of seconds: a value that clearly names one, such as1.0or"1", is used as that integer, and anything else (0.5,"1.5",True,"high") is rejected rather than rounded. A callback attached while a sample is running takes effect from the next sample.The options can be given with the callback, or to use this method as a decorator:
@sim.daily_callback(priority=-5, skip_burn=True, name="nogo") def count_nogo(sim, days_prior): ...- Parameters:
- callback : Callable[[Simulation, int], dict | None] or GenericTracer, optional¶
The callback function to register. It should accept two arguments, which will be the Simulation object and the days_prior, and return a dictionary of interesting things to store, or nothing. Omit it to use this method as a decorator with options.
- priority : int, optional¶
Offset in seconds from the overnight trigger time; see above.
- skip_burn : bool, optional¶
If True, the callback is not called during burn-in samples.
- name : str, optional¶
Store returned dictionaries under
callback_data.<name>instead of the shareddailykey, so they cannot collide with other callbacks’ keys; for a tracer, its result name.
- Returns:
Callable or GenericTracer – The callback, or the fresh copy of a tracer. Without callback, a decorator that registers the function it decorates.
- Raises:
ValueError – If the priority is zero or not a whole number of seconds, the name is reserved or used by a tracer, or the callback is a tracer declaring a hook other than
daily.
- property daily_callbacks : tuple[Callable, ...]¶
Every callback attached to the
dailyhook, in the order they run.Includes RM systems and configured callbacks. This is a read-only snapshot: use the
daily_callbackmethod to attach a callback.
-
end_sample_callback(callback: Callable[[CallbackMixin], dict | None] | GenericTracer | None =
None, *, priority: int | None =None, skip_burn: bool | None =None, name: str | None =None) Callable[[CallbackMixin], dict | None] | GenericTracer¶ Register a function to be triggered at the end of each sample.
The callback function will be triggered before counters are reset or history buffers are rolled over.
It runs at an offset, in seconds, from the trigger time of the final DCP (the day of departure). The offset is the priority option if given, else the callback’s own
priorityattribute if it has one, and otherwise one second, which is after the final DCP’s RM systems (-1) and bookkeeping (0). A negativepriorityruns it before that bookkeeping, or, below -1, before the RM systems as well. Apriorityof zero is not allowed, as for a daily callback.- Parameters:
- callback : Callable[[Simulation], dict | None] or GenericTracer, optional¶
The callback function to register. It should accept a single argument, which will be the Simulation object, and return a dictionary of interesting things to store, or nothing. Omit it to use this method as a decorator with options.
- priority : int, optional¶
Offset in seconds from the final DCP’s trigger time; see above.
- skip_burn : bool, optional¶
If True, the callback is not called during burn-in samples.
- name : str, optional¶
Store returned dictionaries under
callback_data.<name>instead of the shared hook key; for a tracer, its result name.
- Returns:
Callable or GenericTracer – The callback, or the fresh copy of a tracer. Without callback, a decorator that registers the function it decorates.
- property end_sample_callbacks : tuple[Callable, ...]¶
Every callback attached to the
end_samplehook, in the order they run.Includes RM systems and configured callbacks. This is a read-only snapshot: use the
end_sample_callbackmethod to attach a callback.
-
finalize_sample_callback(callback: Callable[[CallbackMixin], dict | None] | GenericTracer | None =
None, *, skip_burn: bool | None =None, name: str | None =None) Callable[[CallbackMixin], dict | None] | GenericTracer¶ Register a synchronous callback after sample bookkeeping finishes.
Callbacks run after departure statistics, history rollover, database commit, and other bookkeeping updates. They observe post-bookkeeping state and cannot change statistics already recorded. Use
end_sampleto access pre-bookkeeping state. Their execution is included in sample timing, before diagnostics are published. They run only on successful completion, not as exception cleanup; callback failures propagate after bookkeeping has completed. Configured callbacks run first, followed by manual callbacks, each in registration order. Event priority does not apply. Exceptions propagate.- Parameters:
- callback : Callable[[Simulation], dict | None] or GenericTracer, optional¶
Callback receiving the simulation. Returned dictionaries are stored under
callback_data.finalize_samplewith trial/sample keys. Omit it to use this method as a decorator with options.- skip_burn : bool, optional¶
If True, the callback is not called during burn-in samples.
- name : str, optional¶
Store returned dictionaries under
callback_data.<name>instead of the shared hook key; for a tracer, its result name.
- Returns:
Callable or GenericTracer – The callback, or the fresh copy of a tracer. Without callback, a decorator that registers the function it decorates.
- property finalize_sample_callbacks : tuple[Callable, ...]¶
Every callback attached to the
finalize_samplehook, in the order they run.Includes RM systems and configured callbacks. This is a read-only snapshot: use the
finalize_sample_callbackmethod to attach a callback.
-
initialize_sample_callback(callback: Callable[[CallbackMixin], dict | None] | GenericTracer | None =
None, *, skip_burn: bool | None =None, name: str | None =None) Callable[[CallbackMixin], dict | None] | GenericTracer¶ Register a synchronous callback before sample demand generation.
The sample number, random seed, and counters have been initialized, but the new sample’s RM steps have not run. Changes to demand inputs affect this sample. Callbacks run in registration order, with configured callbacks first; event priority does not apply.
- Parameters:
- callback : Callable[[Simulation], dict | None] or GenericTracer, optional¶
Callback receiving the simulation. Returned dictionaries are stored under
callback_data.initialize_samplewith trial/sample keys. Omit it to use this method as a decorator with options.- skip_burn : bool, optional¶
If True, the callback is not called during burn-in samples.
- name : str, optional¶
Store returned dictionaries under
callback_data.<name>instead of the shared hook key; for a tracer, its result name.
- Returns:
Callable or GenericTracer – The callback, or the fresh copy of a tracer. Without callback, a decorator that registers the function it decorates.
- property initialize_sample_callbacks : tuple[Callable, ...]¶
Every callback attached to the
initialize_samplehook, in the order they run.Includes RM systems and configured callbacks. This is a read-only snapshot: use the
initialize_sample_callbackmethod to attach a callback.
-
remove_callback(callback: Callable | GenericTracer | str, *, missing_ok: bool =
False) int¶ Detach manually attached callbacks or tracers.
- Parameters:
- callback : Callable or GenericTracer or str¶
What to remove:
a function (or other callable) previously attached, with or without options, is removed from every hook it is attached to; a bound method matches when it binds the same function to the same object, even if it is a new
obj.methodexpression;a tracer, or a tracer’s result name, removes the attached tracer of that name from all its hooks;
a name given to callbacks with
name=removes those callbacks.
- missing_ok : bool, default False¶
Return 0 instead of raising if nothing matches.
- Returns:
int – The number of hook registrations removed.
- Raises:
ValueError – If nothing matches and missing_ok is False. Callbacks built from
config.callbacksand carrier RM systems cannot be removed this way: disable a configured callback in the configuration instead.
Notes
A callback removed while a sample is running may still be called for the rest of that sample; the removal takes effect from the next one.
-
run(use_existing: UseExistingT | dict[str, UseExistingT] =
True, *, tag: str | None =None, check_versions: bool =True, check_content: bool =True, single_process: bool =False, retain_sims: bool =False, write_report: PathLike | bool | None =True, cache_results: bool =True) contrast.Contrast | GenericSimulationTables[source]¶ Run the experiments, choosing sequential or parallel execution automatically.
When
single_process=Truethe experiments are run in sequence via_run_experiments_in_sequence(); otherwise they are dispatched to the parallel job executor via_run_together().- Parameters:
- use_existing : Literal[True, False, "ignore", "raise"] or dict, default True¶
Single value applied to all experiments, or a mapping from experiment tag to a per-experiment value.
True– load from an existing output file if one is found, otherwise run the simulation.False– always run the simulation."ignore"– load from an existing output file if found, otherwise skip the experiment entirely."raise"– raise an error if the output file is missing.
- tag : str, optional¶
If provided, run only the experiment with this tag and return its result directly rather than a
Contrast.- check_versions : bool, default True¶
If True, re-run the simulation when a loaded summary was produced by a different PassengerSim version.
- check_content : bool, default True¶
If True, re-run the simulation when a loaded summary’s config differs from the current config.
- single_process : bool, default False¶
If True, force all experiments to run in single-process mode and execute them sequentially.
- retain_sims : bool, default False¶
If True, keep simulation objects in
simsafter completion. Primarily useful for debugging.- write_report : path-like or bool or None, default True¶
If truthy, write an HTML report when all experiments finish. Pass a path-like value to specify the destination;
Trueuses the default filename"experiments-summary.html". Ignored whenoutput_dir=False.- cache_results : bool, default True¶
If True, cache each experiment’s result on the corresponding
Experimentobject so it can be reused without reloading.
- Returns:
contrast.Contrast or GenericSimulationTables – A
Contrastmapping tags to results, or a single result whentagselects exactly one experiment.
- use_registered_callbacks() None¶
Adopt all globally registered callbacks (deprecated).
Warning
This method emits a DeprecationWarning. For reusable callbacks, register them with
register_callback(functions) orregister_tracer(tracer classes) and select them inconfig.callbacks; those register for configuration rather than attaching to every simulation. For development and testing, attach callbacks directly with the simulation’s hook methods, or use an explicitCallbackMixinbundle andbundle.apply_callback_functions(sim). Registration APIs, including the factory ones, are not deprecated.
- property report_filename : Path¶
Path of the HTML report written after the most recent run.
Unless reporting is disabled, a report is written to a file after
run()completes. The resulting path is stored here.- Returns:
pathlib.Path – The path of the written report file.
- Raises:
ValueError – If no report has been written yet.
- validate() None[source]¶
Validate all experiment tags and callables against the base config.
Checks that every experiment has a unique tag and that its config-transforming callable runs without error when given a deep copy of the base config. Does not verify that modified configs are mutually compatible.
- Returns:
None
- Raises:
ValueError – If any experiment is missing a tag, has a duplicate tag, or its callable raises an exception against the base config.