Tracing¶

In addition to the completely flexible callback data storage system, PassengerSim also includes the ability to “trace” certain details of the simulation. Traces are more aggregate than other callbacks, as they will follow the average values of various measures over many samples. Unlike other aggregate measures, they allow for the selection of a limited number of micro-level details, e.g. path forecasts for specific paths, or bid prices on specific legs. This contrasts with the database functionality, which can (relatively) efficiently store this information for all paths or legs. Tracing allows the analyst to probe a simulation for details of interest without becoming bogged down in the recording and storage of massive amounts of data that really isn’t needed.

import passengersim as pax

pax.versions()

passengersim 0.93
passengersim.core 0.93

Here, we’ll run a quick demo using the “3MKT” example model. We’ll give AL1 the ‘P’ RM system to make it interesting.

cfg = pax.Config.from_yaml(pax.demo_network("3MKT"))

cfg.simulation_controls.num_samples = 400
cfg.simulation_controls.burn_samples = 50
cfg.simulation_controls.num_trials = 1
cfg.db = None
cfg.outputs.reports.clear()
cfg.outputs._write_no_files()

cfg.carriers.AL1.rm_system = "P"

sim = pax.Simulation(cfg)

/Users/jpn/Git/pax/packages/passengersim-public/src/passengersim/config/manipulate.py:49: UserWarning: Only 350 usable samples remain after burning 50 of 400 samples across 1 trials; at least 750 usable samples are recommended to ensure meaningful results.
  schema_validator.validate_python(model.__dict__)

Forecast Tracing¶

The path forecast tracing capabilities allows us to record detailed data about a subset of Simulation path forecasts, so we can review them after the simulation. The tracing will summarize the average forecasts for the specific paths, but will not store the sample-by-sample forecast details, which would be an enormous amount of detail that will require large amounts of memory or disk storage and may not be sufficiently useful for analysis. A similar tracing tool is available for leg forecasts as well.

from passengersim.tracers.forecasts import (
    PathForecastTracer,
    fig_path_forecast_dashboard,
)

f_tracer = PathForecastTracer(path_ids=[1, 5, 9])
f_tracer.attach(sim)

Bid Price Tracing¶

The native summary covering every leg is enabled by default.

After running, access summary.leg_bid_price_history. It is indexed by (leg_id, days_prior) and contains:

summary = pax.Simulation(cfg).run()
daily_prices = summary.leg_bid_price_history
one_leg = daily_prices.xs(101, level="leg_id")

Collection includes each integer day from the earliest configured DCP through departure day, excludes burn-in, and works with cfg.db = None. It is on by default; the table is None when collection is disabled. Set collect_leg_bid_prices: false before the next run to turn it off. The option always collects all legs; filtering the resulting table does not reduce collection. A cell with no observations has a missing mean and standard deviation; a single observation has a missing standard deviation. The native collector stores online statistics rather than individual sample observations, and saved PXSIM and pickle results retain the information needed to combine summaries correctly.

Native collection follows core daily-statistics timing and the existing leg reporting-price convention, including sold-out legs. For positive days_prior, recording occurs at the core daily or DCP event; later callbacks can subsequently change prices. At days_prior == 0, recording occurs at the end of the sample, after the event queue finishes. The native collector has no callback priority setting, so a Python tracer may observe different prices if callbacks change them between these times. Multi-cabin legs record their leg-level reporting price without selecting a cabin or applying cabin capacity-sharing rules.

Each row pools individual observations across retained samples, trials, and workers. The standard deviation is the sample standard deviation of those observations (ddof=1), including variation between trials. It is not the average of trial standard deviations. Rerunning a simulation starts a fresh history; saved summaries retain their previous results. This table requires no database report selection.

You can use native collection alongside LegBidPriceTracer. Use the Python tracer below when selecting only some legs or developing custom collection behavior. Its existing aggregation behavior is unchanged. Cabin-level collection is not yet enabled by this option.

Selected legs and paths¶

Similarly, the bid price tracing feature allows us to record detailed data about a subset of Simulation bid prices. We can record bid prices by leg, path, or both if desired. Each tracer only attaches to the bid prices on a selected subset of individual legs, so as to not overwhelm the simulation with data.

from passengersim.tracers.bid_price import (
    LegBidPriceTracer,
    PathBidPriceTracer,
    fig_leg_bid_prices,
    fig_path_bid_prices,
)

bp_tracer = PathBidPriceTracer(path_ids=[1, 5, 9])
bp_tracer.attach(sim)

leg_bp_tracer = LegBidPriceTracer(leg_ids=[101, 111])
leg_bp_tracer.attach(sim)

Once the tracers have been attached, we run the simulation as normal. The attached tracers will automatically collect and aggregate the relevant data, and attach the tabulated results to the summary outputs.

summary = sim.run()

Task Completed after 4.03 seconds

All the usual summary data remains available for review and analysis.

summary.fig_carrier_revenues()

summary.fig_fare_class_mix()

Dashboards¶

In addition to the usual summary reports, the output summary also provides the data needed to power detailed dashboard visualizations for each of the traced paths or legs.

For the selected paths or legs, we can review a forecast dashboard that shows the forecast mean and std dev for each fare class from each DCP through departure, the mean forecast within each timeframe, as well as history data on yieldable and (if recorded separately) priceable sales, and average closure rates at each DCP. The displayed data in each dashboard is specific to the selected path or leg, but averaged across all relevant (non-burned) samples in the simulation.

fig_path_forecast_dashboard(summary, path_id=1)

fig_path_forecast_dashboard(summary, path_id=5)

The dashboards for bid price tracing are less busy than those for the forecasts, as the bid price

fig_path_bid_prices(summary)

fig_leg_bid_prices(summary)

Tracers in Callback Data¶

All the underlying data for these reports is stored in the summary’s callback_data attribute, if you want to access it to parse or visualize it differently. The forecast tracers contain selected summary statistics by path or leg, days prior, and fare class. These statistics are aggregated across all relevant (i.e. non-burned) samples.

summary.callback_data.selected_path_forecasts

The bid prices data includes the mean and standard deviation of the leg or path bid price, by days prior.

summary.callback_data.leg_bid_prices

Relationship to Callbacks¶

Unlike other callback data, the tracers are not stored by sample day, as that would generally create an overwhelming amount of data to store, and we are typically not interested in that much detail. If we are interested in grabbing and storing path forecast data for individual sample days, we can still do that with the regular callback interface.

sim1 = pax.Simulation(cfg)

/Users/jpn/Git/pax/packages/passengersim-public/src/passengersim/config/manipulate.py:49: UserWarning: Only 350 usable samples remain after burning 50 of 400 samples across 1 trials; at least 750 usable samples are recommended to ensure meaningful results.
  schema_validator.validate_python(model.__dict__)

@sim1.begin_sample_callback
def grab_forecasts(sim):
    if sim.eng.sample not in [300, 375]:
        return
    return {
        f"path-{p}": sim.eng.paths.select(path_id=p).get_forecast_data()
        for p in [1, 5, 9]
    }

summary1 = sim1.run()

Task Completed after 2.81 seconds

When run like this, we capture not the average path forecast over the simulation, but rather the exact path forecast for the selected paths (1 and 9) at sample days 300 and 375. The data is stored in the callback_data.begin_sample attribute:

summary1.callback_data.begin_sample

[{'trial': 0,
  'sample': 300,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x1233af260>,
  'path-5': <passengersim_core.forecast_tools.ForecastData at 0x1235d5400>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x123675d60>},
 {'trial': 0,
  'sample': 375,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x123688530>,
  'path-5': <passengersim_core.forecast_tools.ForecastData at 0x123688a70>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x1236ade80>}]

We can review the details of each specific forecast by accessing the dashboard visualization.

summary1.callback_data.begin_sample[0]["path-1"].dashboard()

We can also access individual sub-tables of forecast data as pandas DataFrames, to manipulate or visualize as we like.

summary1.callback_data.begin_sample[0]["path-1"].history_sold_yieldable

summary1.callback_data.begin_sample[0]["path-1"].mean_in_timeframe