Tracing¶

In addition to the completely flexible callback data storage system, PassengerSim also includes the ability to "trace" certain details of the simulation. Traces are more aggregate than other callbacks, as they will follow the average values of various measures over many samples. Unlike other aggregate measures, they allow for the selection of a limited number of micro-level details, e.g. path forecasts for specific paths, or bid prices on specific legs. This contrasts with the database functionality, which can (relatively) efficiently store this information for all paths or legs. Tracing allows the analyst to probe a simulation for details of interest without becoming bogged down in the recording and storage of massive amounts of data that really isn't needed.

In [1]:

import passengersim as pax

pax.versions()

import passengersim as pax pax.versions()

passengersim 0.58
passengersim.core 0.59.dev1+g671876c7d

Here, we'll run a quick demo using the "3MKT" example model. We'll give AL1 the 'P' RM system to make it interesting.

In [2]:

cfg = pax.Config.from_yaml(pax.demo_network("3MKT"))

cfg.simulation_controls.num_samples = 400
cfg.simulation_controls.burn_samples = 50
cfg.simulation_controls.num_trials = 1
cfg.db = None
cfg.outputs.reports.clear()

cfg.carriers.AL1.rm_system = "P"

sim = pax.Simulation(cfg)

cfg = pax.Config.from_yaml(pax.demo_network("3MKT")) cfg.simulation_controls.num_samples = 400 cfg.simulation_controls.burn_samples = 50 cfg.simulation_controls.num_trials = 1 cfg.db = None cfg.outputs.reports.clear() cfg.carriers.AL1.rm_system = "P" sim = pax.Simulation(cfg)

Forecast Tracing¶

The path forecast tracing capabilities allows us to record detailed data about a subset of Simulation path forecasts, so we can review them after the simulation. The tracing will summarize the average forecasts for the specific paths, but will not store the sample-by-sample forecast details, which would be an enormous amount of detail that will require large amounts of memory or disk storage and may not be sufficiently useful for analysis. A similar tracing tool is available for leg forecasts as well.

In [3]:

from passengersim.tracers.forecasts import (
    PathForecastTracer,
    fig_path_forecast_dashboard,
)

f_tracer = PathForecastTracer(path_ids=[1, 9])
f_tracer.attach(sim)

from passengersim.tracers.forecasts import ( PathForecastTracer, fig_path_forecast_dashboard, ) f_tracer = PathForecastTracer(path_ids=[1, 9]) f_tracer.attach(sim)

Bid Price Tracing¶

Similarly, the bid price tracing feature allows us to record detailed data about a subset of Simulation bid prices. We can record bid prices by leg, path, or both if desired. Each tracer only attaches to the bid prices on a selected subset of individual legs, so as to not overwhelm the simulation with data.

In [4]:

from passengersim.tracers.bid_price import (
    LegBidPriceTracer,
    PathBidPriceTracer,
    fig_leg_bid_prices,
    fig_path_bid_prices,
)

bp_tracer = PathBidPriceTracer(path_ids=[1, 9])
bp_tracer.attach(sim)

leg_bp_tracer = LegBidPriceTracer(leg_ids=[101, 111])
leg_bp_tracer.attach(sim)

from passengersim.tracers.bid_price import ( LegBidPriceTracer, PathBidPriceTracer, fig_leg_bid_prices, fig_path_bid_prices, ) bp_tracer = PathBidPriceTracer(path_ids=[1, 9]) bp_tracer.attach(sim) leg_bp_tracer = LegBidPriceTracer(leg_ids=[101, 111]) leg_bp_tracer.attach(sim)

Once the tracers have been attached, we run the simulation as normal. The attached tracers will automatically collect and aggregate the relevant data, and attach the tabulated results to the summary outputs.

In [5]:

summary = sim.run()

summary = sim.run()

Task Completed after 3.17 seconds

All the usual summary data remains available for review and analysis.

In [6]:

summary.fig_carrier_revenues()

summary.fig_carrier_revenues()

Out[6]:

In [7]:

summary.fig_fare_class_mix()

summary.fig_fare_class_mix()

Out[7]:

Dashboards¶

In addition to the usual summary reports, the output summary also provides the data needed to power detailed dashboard visualizations for each of the traced paths or legs.

For the selected paths or legs, we can review a forecast dashboard that shows the forecast mean and std dev for each fare class from each DCP through departure, the mean forecast within each timeframe, as well as history data on yieldable and (if recorded separately) priceable sales, and average closure rates at each DCP. The displayed data in each dashboard is specific to the selected path or leg, but averaged across all relevant (non-burned) samples in the simulation.

In [8]:

fig_path_forecast_dashboard(summary, path_id=1)

fig_path_forecast_dashboard(summary, path_id=1)

Out[8]:

In [9]:

fig_path_forecast_dashboard(summary, path_id=9)

fig_path_forecast_dashboard(summary, path_id=9)

Out[9]:

The dashboards for bid price tracing are less busy than those for the forecasts, as the bid price

In [10]:

fig_path_bid_prices(summary)

fig_path_bid_prices(summary)

Out[10]:

In [11]:

fig_leg_bid_prices(summary)

fig_leg_bid_prices(summary)

Out[11]:

Tracers in Callback Data¶

All the underlying data for these reports is stored in the summary's callback_data attribute, if you want to access it to parse or visualize it differently. The forecast tracers contain selected summary statistics by path or leg, days prior, and fare class. These statistics are aggregated across all relevant (i.e. non-burned) samples.

In [12]:

summary.callback_data.selected_path_forecasts

summary.callback_data.selected_path_forecasts

Out[12]:

		mean_to_departure						stdev_to_departure				...	history_sold_yieldable				history_closure
	booking_class	Y0	Y1	Y2	Y3	Y4	Y5	Y0	Y1	Y2	Y3	...	Y2	Y3	Y4	Y5	Y0	Y1	Y2	Y3	Y4	Y5
path_id	days_prior
1	63	2.542146	9.941561	5.584702	1.794732	11.739468	6.291431	1.771631	4.319455	2.861782	1.579957	...	0.351978	0.069780	2.359121	1.333736	0.000000	0.000000	0.000000	0.000000	0.000000	0.027802
	56	2.436102	9.366506	5.232724	1.724952	9.380347	4.931511	1.714093	4.132766	2.724080	1.530989	...	0.289121	0.038571	1.079451	0.620330	0.000000	0.000000	0.000000	0.000000	0.002857	0.044945
	49	2.344673	8.986836	4.943603	1.686380	8.296376	4.267961	1.679260	4.066012	2.619407	1.496378	...	0.255385	0.038022	1.323297	0.749231	0.000000	0.000000	0.000000	0.000000	0.002857	0.045275
	42	2.234673	8.614418	4.688219	1.648358	6.967735	3.478519	1.647875	3.945773	2.504475	1.477896	...	0.304615	0.051209	1.186264	0.723736	0.000000	0.000000	0.000000	0.000000	0.005714	0.079780
	35	2.118300	8.185957	4.383603	1.597149	5.774734	2.682167	1.604271	3.696554	2.417460	1.460541	...	0.187473	0.024725	0.755495	0.475934	0.000000	0.000000	0.000000	0.000000	0.008901	0.106044
	31	2.059618	7.866726	4.196131	1.572424	5.013302	2.128337	1.595658	3.609585	2.374704	1.455211	...	0.111538	0.024286	0.571099	0.354286	0.000000	0.000000	0.000000	0.000000	0.016484	0.124176
	28	2.023465	7.695407	4.084592	1.548138	4.430047	1.697790	1.576588	3.560280	2.331070	1.451886	...	0.144945	0.022198	1.129670	0.678242	0.000000	0.000000	0.000000	0.000000	0.044725	0.159780
	24	1.955772	7.378264	3.939647	1.525941	3.250958	0.865343	1.544081	3.428452	2.314346	1.440032	...	0.140440	0.045385	1.117473	0.694286	0.000000	0.000000	0.000000	0.011429	0.067692	0.181538
	21	1.869728	7.110462	3.799208	1.475321	2.053770	0.000000	1.522651	3.363088	2.267855	1.383390	...	0.260330	0.072418	0.883297	0.000000	0.000000	0.000000	0.004725	0.043736	0.116154	1.000000
	17	1.786871	6.718924	3.537032	1.380092	1.042731	0.000000	1.461959	3.230329	2.197576	1.273061	...	0.233297	0.057802	0.856484	0.000000	0.000000	0.000000	0.021868	0.093297	0.147582	1.000000
	14	1.664893	6.293869	3.290389	1.293480	0.000000	0.000000	1.400177	3.072104	2.020656	1.187915	...	0.652088	0.611429	0.000000	0.000000	0.000000	0.002857	0.065055	0.178242	1.000000	1.000000
	10	1.454454	5.449463	2.583637	0.515139	0.000000	0.000000	1.268630	2.711183	1.764790	0.635040	...	0.685495	0.375165	0.000000	0.000000	0.000000	0.021868	0.126484	0.200000	1.000000	1.000000
	7	1.190937	4.441360	1.778410	0.000000	0.000000	0.000000	1.137997	2.322363	1.379983	0.000000	...	0.513516	0.000000	0.000000	0.000000	0.008571	0.058571	0.161319	1.000000	1.000000	1.000000
	5	0.960961	3.719145	1.123151	0.000000	0.000000	0.000000	1.032368	2.000465	1.069064	0.000000	...	0.865714	0.000000	0.000000	0.000000	0.052967	0.115714	0.199451	1.000000	1.000000	1.000000
	3	0.758311	2.893502	0.000000	0.000000	0.000000	0.000000	0.866083	1.679374	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.179560	0.196703	1.000000	1.000000	1.000000	1.000000
	1	0.177835	0.735596	0.000000	0.000000	0.000000	0.000000	0.340493	0.743147	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.202747	0.217033	1.000000	1.000000	1.000000	1.000000
9	63	11.095695	7.052048	6.390467	3.957610	21.942445	11.077824	4.726294	3.652488	3.625659	3.685562	...	0.287033	0.065055	3.847363	0.900110	0.000000	0.000000	0.000000	0.000000	0.009670	0.695714
	56	10.500421	6.765455	6.103434	3.892555	18.076396	9.320925	4.468967	3.498517	3.523963	3.675231	...	0.230000	0.062967	1.758022	0.435275	0.000000	0.000000	0.000000	0.000000	0.018242	0.637253
	49	10.026904	6.520729	5.873434	3.829588	16.291918	8.256781	4.348219	3.353806	3.462947	3.665369	...	0.205714	0.048571	2.067363	0.581648	0.000000	0.000000	0.000000	0.000000	0.027802	0.647912
	42	9.623278	6.294246	5.667720	3.781016	14.182168	6.913298	4.198069	3.266205	3.400609	3.638974	...	0.230220	0.087692	2.177143	0.639121	0.000000	0.000000	0.000000	0.000000	0.062088	0.606923
	35	9.105256	5.950400	5.437500	3.693324	11.882819	5.375649	4.034654	3.145313	3.353641	3.581604	...	0.130879	0.046593	1.293407	0.495275	0.000000	0.000000	0.000000	0.000000	0.079231	0.591319
	31	8.812289	5.790729	5.306621	3.646730	10.478225	4.109598	3.942200	3.118391	3.269516	3.515460	...	0.068791	0.043846	0.988132	0.348462	0.000000	0.000000	0.000000	0.000000	0.102088	0.540110
	28	8.566684	5.690949	5.237830	3.602884	9.349790	3.188522	3.896806	3.066985	3.265444	3.475829	...	0.161648	0.130769	1.987363	0.779231	0.000000	0.000000	0.000000	0.000000	0.158022	0.500989
	24	8.212289	5.521828	5.076181	3.472115	7.015424	1.551609	3.824400	3.013752	3.191924	3.287509	...	0.148791	0.310220	1.916593	0.809121	0.000000	0.000000	0.000000	0.017143	0.225714	0.485824
	21	7.919871	5.343147	4.927390	3.127391	4.601234	0.000000	3.746274	2.962048	3.155510	2.773445	...	0.208901	0.332637	1.701868	0.000000	0.000000	0.000000	0.000000	0.065934	0.289231	1.000000
	17	7.480421	5.143696	4.718489	2.732468	2.248740	0.000000	3.550376	2.926404	3.078314	2.199131	...	0.330989	0.396484	1.478571	0.000000	0.000000	0.000000	0.005714	0.141099	0.368352	1.000000
	14	6.975915	4.869301	4.380354	2.184727	0.000000	0.000000	3.394130	2.837693	2.824929	1.512106	...	0.816154	1.009780	0.000000	0.000000	0.000000	0.000000	0.042857	0.224835	1.000000	1.000000
	10	5.934816	4.322597	3.520753	0.865754	0.000000	0.000000	2.978688	2.600283	2.501777	0.801162	...	0.897363	0.594615	0.000000	0.000000	0.000000	0.014286	0.129780	0.289560	1.000000	1.000000
	7	4.733717	3.613382	2.468508	0.000000	0.000000	0.000000	2.452147	2.283770	1.817399	0.000000	...	0.811978	0.000000	0.000000	0.000000	0.005714	0.036593	0.182198	1.000000	1.000000	1.000000
	5	3.880456	3.003979	1.433578	0.000000	0.000000	0.000000	2.089969	1.930955	1.232235	0.000000	...	1.078791	0.000000	0.000000	0.000000	0.044835	0.102527	0.247912	1.000000	1.000000	1.000000
	3	2.940177	2.384334	0.000000	0.000000	0.000000	0.000000	1.608757	1.583988	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.315824	0.315824	1.000000	1.000000	1.000000	1.000000
	1	0.708996	0.712860	0.000000	0.000000	0.000000	0.000000	0.649894	0.749299	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.390330	0.390330	1.000000	1.000000	1.000000	1.000000

32 rows × 36 columns

The bid prices data includes the mean and standard deviation of the leg or path bid price, by days prior.

In [13]:

summary.callback_data.leg_bid_prices

summary.callback_data.leg_bid_prices

Out[13]:

	leg_id	101	111
statistic	days_prior
mean	63	52.591280	152.434345
	62	52.367467	152.140443
	61	53.034745	151.776852
	60	52.925320	152.223692
	59	52.317152	152.609186
...	...	...	...
std_dev	4	115.335364	171.875792
	3	127.414644	180.136785
	2	141.228146	163.284747
	1	130.021006	161.918451
	0	130.021006	161.918451

128 rows × 2 columns

Relationship to Callbacks¶

Unlike other callback data, the tracers are not stored by sample day, as that would generally create an overwhelming amount of data to store, and we are typically not interested in that much detail. If we are interested in grabbing and storing path forecast data for individual sample days, we can still do that with the regular callback interface.

In [14]:

sim1 = pax.Simulation(cfg)

sim1 = pax.Simulation(cfg)

In [15]:

@sim1.begin_sample_callback
def grab_forecasts(sim):
    if sim.eng.sample not in [300, 375]:
        return
    return {f"path-{p}": sim.eng.paths.select(path_id=p).get_forecast_data() for p in [1, 9]}

@sim1.begin_sample_callback def grab_forecasts(sim): if sim.eng.sample not in [300, 375]: return return {f"path-{p}": sim.eng.paths.select(path_id=p).get_forecast_data() for p in [1, 9]}

In [16]:

summary1 = sim1.run()

summary1 = sim1.run()

Task Completed after 1.96 seconds

When run like this, we capture not the average path forecast over the simulation, but rather the exact path forecast for the selected paths (1 and 9) at sample days 300 and 375. The data is stored in the callback_data.begin_sample attribute:

In [17]:

summary1.callback_data.begin_sample

summary1.callback_data.begin_sample

Out[17]:

[{'trial': 0,
  'sample': 300,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x13876eb10>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x138b8cbf0>},
 {'trial': 0,
  'sample': 375,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x138bb4bf0>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x138bb4950>}]

We can review the details of each specific forecast by accessing the dashboard visualization.

In [18]:

summary1.callback_data.begin_sample[0]["path-1"].dashboard()

summary1.callback_data.begin_sample[0]["path-1"].dashboard()

Out[18]:

We can also access individual sub-tables of forecast data as pandas DataFrames, to manipulate or visualize as we like.

In [19]:

summary1.callback_data.begin_sample[0]["path-1"].history_sold_yieldable

summary1.callback_data.begin_sample[0]["path-1"].history_sold_yieldable

Out[19]:

booking_class	Y0	Y1	Y2	Y3	Y4	Y5
tf_index
0	0.192308	0.500000	0.307692	0.038462	1.923077	1.307692
1	0.153846	0.538462	0.307692	0.000000	0.576923	0.653846
2	0.076923	0.153846	0.269231	0.000000	1.192308	0.576923
3	0.115385	0.384615	0.153846	0.076923	0.961538	0.692308
4	0.038462	0.346154	0.153846	0.038462	0.692308	0.346154
5	0.038462	0.076923	0.115385	0.076923	0.692308	0.269231
6	0.038462	0.384615	0.230769	0.000000	1.000000	0.615385
7	0.000000	0.192308	0.115385	0.115385	1.307692	0.769231
8	0.230769	0.384615	0.384615	0.000000	0.923077	0.000000
9	0.076923	0.307692	0.269231	0.000000	1.115385	0.000000
10	0.192308	0.923077	0.500000	0.576923	0.000000	0.000000
11	0.230769	0.730769	0.461538	0.153846	0.000000	0.000000
12	0.153846	0.269231	0.576923	0.000000	0.000000	0.000000
13	0.153846	0.576923	0.653846	0.000000	0.000000	0.000000
14	0.269231	0.961538	0.000000	0.000000	0.000000	0.000000
15	0.038462	0.307692	0.000000	0.000000	0.000000	0.000000

In [20]:

summary1.callback_data.begin_sample[0]["path-1"].mean_in_timeframe

summary1.callback_data.begin_sample[0]["path-1"].mean_in_timeframe

Out[20]:

booking_class	Y0	Y1	Y2	Y3	Y4	Y5
tf_index
0	0.192308	0.500000	0.307692	0.038462	1.923077	1.307692
1	0.153846	0.538462	0.307692	0.000000	0.576923	0.653846
2	0.076923	0.153846	0.269231	0.000000	1.192308	0.576923
3	0.115385	0.384615	0.153846	0.076923	0.961538	0.692308
4	0.038462	0.346154	0.153846	0.038462	0.692308	0.346154
5	0.038462	0.076923	0.115385	0.076923	0.692308	0.290420
6	0.038462	0.384615	0.230769	0.000000	1.000000	0.676087
7	0.000000	0.192308	0.115385	0.115385	1.307692	0.855903
8	0.230769	0.384615	0.384615	0.000000	1.048620	0.000000
9	0.076923	0.307692	0.269231	0.000000	1.311751	0.000000
10	0.192308	0.923077	0.544670	0.740127	0.000000	0.000000
11	0.230769	0.730769	0.562779	0.247132	0.000000	0.000000
12	0.153846	0.359422	0.766563	0.000000	0.000000	0.000000
13	0.167882	0.711566	0.847891	0.000000	0.000000	0.000000
14	0.397941	1.222805	0.000000	0.000000	0.000000	0.000000
15	0.084310	0.438919	0.000000	0.000000	0.000000	0.000000