Lab 3: Exploring Outputs¶

Does it say “notebook is read only” in a button on your toolbar? Is the “save” button greyed out?
If you want to be able to run this notebook and save your work, you need to copy it to your user directory. In the file browser on the left (click the folder icon), right click this notebook file (lab-1.ipynb), copy it, then navigate to your user home directory, and right click and choose paste. Then open that copy instead of the read-only copy, and you’ll be good to go.

import altair as alt
import pandas as pd

import passengersim as pax

pax.versions()

passengersim 0.80
passengersim.core 0.80

In this lab, we will look at explore some PassengerSim outputs in detail. To get started, we’ll load the config for the standard 3MKT demo model to work with. We’ll also assign the two carriers RM systems that will be more interesting to work with.

cfg = pax.Config.from_yaml(pax.demo_network("3MKT/DEMO"))

cfg.carriers.AL1.rm_system = "U"
cfg.carriers.AL2.rm_system = "P"

sim = pax.MultiSimulation(cfg)
summary = sim.run()

   ╭─────────────────────────────────────╮                                                                         
   │ licensed for jpn                    │                                                                         
   │ until 2036-05-28 15:49:32+00:00 UTC │                                                                         
   │ maximum 42000 legs in network       │                                                                         
   ╰─────────────────────────────────────╯                                                                         

Included Data¶

summary

<passengersim.summaries.SimulationTables created on 2026-06-12>
 * bid_price_history (256 row DataFrame)
 * cabins (5600 row DataFrame)
 * carriers (2 row DataFrame)
 * carrier_history (NoneType)
 * carrier_history2 (1400 row DataFrame)
 * forecast_accuracy (NoneType)
 * cp_segmentation (12 row DataFrame)
 * demand_to_come_summary (34 row DataFrame)
 * demand_to_come (NoneType)
 * demands (6 row DataFrame)
 * demand_history (NoneType)
 * displacement_history (68 row DataFrame)
 * fare_class_mix (12 row DataFrame)
 * path_forecasts (NoneType)
 * leg_forecasts (NoneType)
 * edgar (NoneType)
 * legbuckets (48 row DataFrame)
 * legs (8 row DataFrame)
 * leg_detail (NoneType)
 * local_and_flow_yields (NoneType)
 * pathclasses (72 row DataFrame)
 * path_legs (16 row DataFrame)
 * paths (12 row DataFrame)
 * segmentation_by_timeframe (336 row DataFrame)
 * segmentation_detail (NoneType)
<*>

There are lots of visualizations we can make from the summary output. The figures you can access are all prefixed with “fig”.

High Level Aggregate Measures¶

There’s a bunch of high level aggregate outputs that can be visualized. It’s convenient to have these and easy to script up a much that you’re interested in.

summary.fig_carrier_revenues()

If you are working in a Jupyter notebook, each cell will display the output from the last line of code in the cell, so you can put one figure in each cell and run like that.

Or, since these pre-canned figures are made using altair, you can call the .show() method on them, and put a bunch all in one cell.

summary.fig_carrier_revenues().show()
summary.fig_carrier_load_factors().show()
summary.fig_carrier_local_share().show()
summary.fig_carrier_total_bookings().show()
summary.fig_carrier_rasm().show()

All these carrier aggregate measures are backed up by data in the carriers dataframe on the summary.

summary.carriers

You don’t have to use altair for visualization, you can take the underlying data and visualize it however you like, with whatever Python libraries you’re comfortable with. You can even make the figures super misleading, if that suits your fancy!

import matplotlib.pyplot as plt

plt.bar(summary.carriers.index, summary.carriers["avg_rev"])
plt.xlabel("Carrier")
plt.ylabel("Average Revenue")
plt.title("Average Revenue by Carrier")
plt.ylim(summary.carriers["avg_rev"].min() - 100, summary.carriers["avg_rev"].max() + 100)
plt.show()

Bookings by Timeframe¶

summary.fig_bookings_by_timeframe()

If use the by_carrier argument to get the details for only a single carrier, PassengerSim will automatically enhance the detail of the figure to drill down into passenger segments.

summary.fig_bookings_by_timeframe(by_carrier="AL1")

We can further drill down by fare classes. This will preserve the passenger segmentation, not as the bar colors but in seperate facet panels.

summary.fig_bookings_by_timeframe(by_carrier="AL1", by_class=True)

Underneath these figures is a segmentation by timeframe dataframe, which has data we can slice and dice in different ways.

summary.segmentation_by_timeframe

Bid Prices¶

We also have a variety of bid price data that is tracked automatically, and which we can examine. The bid price history shows how the average bid prices evolve over time.

summary.fig_bid_price_history()

There are some technical oddities in the average bid prices. Most notably, in the absence of overbooking the bid price isn’t defined for flights that are sold out. The history visualized here uses a placeholder value for those sold out flights: the bid price of the last seat, the moment before it was sold. To show only the average bid prices for legs where there is some capacity remaining, we can specify cap="some", and we will get a picture that looks very similar, until the final few days prior to departure.

summary.fig_bid_price_history(cap="some")

Other Included Data¶

There is quite a variety of data in the summary object. You can see what’s available as a list in the repr that displays for the summary.

summary

<passengersim.summaries.SimulationTables created on 2026-06-12>
 * bid_price_history (256 row DataFrame)
 * cabins (5600 row DataFrame)
 * carriers (2 row DataFrame)
 * carrier_history (NoneType)
 * carrier_history2 (1400 row DataFrame)
 * forecast_accuracy (NoneType)
 * cp_segmentation (12 row DataFrame)
 * demand_to_come_summary (34 row DataFrame)
 * demand_to_come (NoneType)
 * demands (6 row DataFrame)
 * demand_history (NoneType)
 * displacement_history (68 row DataFrame)
 * fare_class_mix (12 row DataFrame)
 * path_forecasts (NoneType)
 * leg_forecasts (NoneType)
 * edgar (NoneType)
 * legbuckets (48 row DataFrame)
 * legs (8 row DataFrame)
 * leg_detail (NoneType)
 * local_and_flow_yields (NoneType)
 * pathclasses (72 row DataFrame)
 * path_legs (16 row DataFrame)
 * paths (12 row DataFrame)
 * segmentation_by_timeframe (336 row DataFrame)
 * segmentation_detail (NoneType)
<*>

There are numerous data tables that have no canned visualizations attached to them, but that should not limit you to want you can look at in the results. Just as you can roll your own visualizations to replace existing ones, you can also do all kinds of novel analysis with the data that’s provided.

For example, let’s look at the data in the pathclasses table.

summary.pathclasses

All the columns prefixed by gt_ represent “grand total” values across all simulation samples (after the burn). You can easily convert those columns to averages by dividing by the number of samples in the data.

pc = summary.pathclasses.filter(regex="^gt_")
pc.columns = pc.columns.str.removeprefix("gt_")
pc /= summary.n_total_samples
pc = summary.pathclasses.join(pc.add_prefix("average_"))
pc

df = (
    pc.query("carrier == 'AL1'")
    .rename(columns={"gt_revenue_by_segment_business": "Business", "gt_revenue_by_segment_leisure": "Leisure"})
    .reset_index()
    .melt(
        id_vars=["path_id", "booking_class"],
        value_vars=["Business", "Leisure"],
        var_name="segment",
        value_name="revenue",
    )
)

alt.Chart(df).mark_bar().encode(
    y="booking_class",
    x="revenue",
    color="segment",
    tooltip=["booking_class", "segment", "revenue"],
).facet(row="path_id")

Collecting Other Data with Callbacks¶

PassengerSim includes a variety of optimized data collection processes that run automatically during a simulation, but these pre-selected data may not be sufficient for every analysis. To supplement this, users can choose to additionally collect any other data while running a simulation. This is done by writing a “callback” function. Such a function is invoked regularly while the simulation is running, and can inspect and store almost anything from the Simulation object.

The primary thing to keep in mind about callbacks is that they are going to run during your simulation, so you have to set them up before running the simulation. You cannot do anything to change the callbacks once the simulation is finished any you have summary result object.

So, to demo the callbacks, we’ll create a new simulation but not run it (yet).

sim = pax.Simulation(cfg)

Types of Callback Functions¶

To collect data, we can write a function that will interrogate the simulation and grab whatever info we are looking for. There are three different points where we can attach data collection callback functions:

• begin_sample, which will trigger data collection at the beginning of each sample, after the RM systems for each carrier are initialized (e.g. with forecasts, etc) but before any customers can arrive.

• end_sample, which will trigger data collection at the end of each sample, after customers have arrive and all bookings have be finalized.

• daily, which will trigger data collection once per day during every sample, just after any DCP or daily RM system updates are run.

The first two callbacks (begin and end sample) are written as a function that accepts one argument (the Simulation object), and either returns nothing (to ignore that event) or returns a dictionary of values to store, where the keys are all strings naming what’s being stored and the values can be whatever is of interest. This can be a simple numeric value (i.e., a scalar), or a tuple, an array, a nested dictionary, or any other pickle-able Python object.

We can attach each callback to the Simulation by using a Python decorator.

Example Callback Functions¶

For example, here we create a callback to collect carrier revenue at the end of every sample. Note that we skip the burn period by returning nothing for those samples; this is not required by the callback algorithm but is good practice for analysis.

@sim.end_sample_callback
def collect_carrier_revenue(sim: pax.Simulation) -> dict | None:
    if sim.eng.sample < sim.eng.burn_samples:
        return
    return {c.name: c.revenue for c in sim.eng.carriers}

The daily callback operates similarly, except it accepts a second argument that gives the number of days prior to departure for this day. You don’t need to use the second argument in the callback function, but you need to including in the function signature (and you can use it if desired, e.g. to collect data only at DCPs instead of every day). In the example here, we collect daily carrier revenue, but only every 7th sample, which is a good way to reduce the overhead from collecting detailed data.

@sim.daily_callback
def collect_carrier_revenue_detail(sim: pax.Simulation, days_prior: int) -> dict | None:
    if sim.eng.sample < sim.eng.burn_samples:
        return
    if sim.eng.sample % 7 == 0:
        return {c.name: c.revenue for c in sim.eng.carriers}

Multiple callbacks of the same kind can be attached (i.e. there can be two end_sample callbacks). The only limitation is that the named values in the return values of each callback function must be unique, or else they will overwrite one another.

For example, suppose we also want to count for each carrier the number of passengers departing each airport on each sample day. The previous end sample callback stored revenue values in a dictionary keyed by carrier name, so if we don’t want to overwrite that, we need to use a different key. One way to avoid that is to just nest the output of the callback function in another dictionary with a unique top level key.

from collections import defaultdict


@sim.end_sample_callback
def collect_passenger_counts(sim: pax.Simulation) -> dict | None:
    if sim.eng.sample < sim.eng.burn_samples:
        return
    paxcount = defaultdict(lambda: defaultdict(int))
    for leg in sim.eng.legs:
        paxcount[leg.carrier.name][leg.orig] += leg.sold
    # convert defaultdict to a regular dict, not necessary but pickles smaller
    paxcount = {carrier: dict(airports) for carrier, airports in paxcount.items()}
    return {"psgr_by_airport": paxcount}

One of the nifty features of callbacks is that they can access anything available in the simulation, not just sales and revenue data from carriers. For example, we can inspect demand objects directly, and see how many potential passengers were simulated so far, and how many didn’t make a booking on any airlines (i.e. the “no-go” customers).

@sim.daily_callback
def count_nogo(sim: pax.Simulation, days_prior: int) -> dict | None:
    if sim.eng.sample < sim.eng.burn_samples:
        return
    if sim.eng.sample % 7 == 0:
        return
    if days_prior > 0 and days_prior not in sim.config.dcps:
        # Only count "nogo" (unsold) demand at DCPs, and at departure (days_prior == 0)
        return
    nogo_count = defaultdict(lambda: defaultdict(lambda: defaultdict(int)))
    for dmd in sim.eng.demands:
        nogo_count[dmd.orig][dmd.dest][dmd.segment] += dmd.unsold
    # convert defaultdict to a regular dict, not necessary but pickles smaller
    nogo_count = {orig: {dest: dict(seg) for dest, seg in dests.items()} for orig, dests in nogo_count.items()}
    return {"nogo": nogo_count}

Re-using Callback Functions¶

Attaching via the decorators is a convenient way to add callbacks to a single simulation. The decorators connect the callback function to the simulation, but do not otherwise modify the function itself. It is easy to define callback functions in a seperate module or to re-use callback functions for multiple simulations, by using the decorator as a regular function. For example, we can create a second simulation object, and attach the same callback functions like this:

duplicate_sim = pax.Simulation(cfg)
duplicate_sim.end_sample_callback(collect_carrier_revenue)
duplicate_sim.daily_callback(collect_carrier_revenue_detail);

In this example, the duplicate_sim is running the same config as the original, but this would work with a modified config or even a completely different network.

Once we have attached all desired callbacks to the simulation we want to run, we can run it as normal.

summary = sim.run()

Task Completed after 9.45 seconds

All the usual summary data remains available for review and analysis.

summary.fig_carrier_revenues()

Callback Data¶

In addition to the usual suspects, the summary object includes the collected callback data from our callback functions.

summary.callback_data

<passengersim.callbacks.base.CallbackData from daily, end_sample>

Because we connected a “daily” callback, the data we collected is available under the callback_data.daily accessor.

summary.callback_data.daily[:5]

[{'trial': 0,
  'sample': 50,
  'days_prior': 63,
  'nogo': {'BOS': {'ORD': {'business': 0, 'leisure': 0},
    'LAX': {'business': 0, 'leisure': 0}},
   'ORD': {'LAX': {'business': 0, 'leisure': 0}}}},
 {'trial': 0,
  'sample': 50,
  'days_prior': 56,
  'nogo': {'BOS': {'ORD': {'business': 0, 'leisure': 0},
    'LAX': {'business': 0, 'leisure': 0}},
   'ORD': {'LAX': {'business': 0, 'leisure': 0}}}},
 {'trial': 0,
  'sample': 50,
  'days_prior': 49,
  'nogo': {'BOS': {'ORD': {'business': 0, 'leisure': 0},
    'LAX': {'business': 0, 'leisure': 0}},
   'ORD': {'LAX': {'business': 0, 'leisure': 0}}}},
 {'trial': 0,
  'sample': 50,
  'days_prior': 42,
  'nogo': {'BOS': {'ORD': {'business': 0, 'leisure': 0},
    'LAX': {'business': 0, 'leisure': 0}},
   'ORD': {'LAX': {'business': 0, 'leisure': 0}}}},
 {'trial': 0,
  'sample': 50,
  'days_prior': 35,
  'nogo': {'BOS': {'ORD': {'business': 0, 'leisure': 0},
    'LAX': {'business': 0, 'leisure': 0}},
   'ORD': {'LAX': {'business': 0, 'leisure': 0}}}}]

As you might expect, the “begin_sample” or “end_sample” callbacks are available under callback_data.begin_sample or callback_data.end_sample, respectively.

summary.callback_data.end_sample[:3]

[{'trial': 0,
  'sample': 50,
  'AL1': 80300.0,
  'AL2': 83425.0,
  'psgr_by_airport': {'AL1': {'BOS': 158.0, 'ORD': 180.0},
   'AL2': {'BOS': 155.0, 'ORD': 195.0}}},
 {'trial': 0,
  'sample': 51,
  'psgr_by_airport': {'AL1': {'BOS': 159.0, 'ORD': 220.0},
   'AL2': {'BOS': 150.0, 'ORD': 193.0}},
  'AL1': 86100.0,
  'AL2': 75200.0},
 {'trial': 0,
  'sample': 52,
  'AL1': 91725.0,
  'AL2': 96200.0,
  'psgr_by_airport': {'AL1': {'BOS': 190.0, 'ORD': 229.0},
   'AL2': {'BOS': 194.0, 'ORD': 223.0}}}]

The callback data can include pretty much anything, so it is stored in a very flexible (but inefficient) format: a list of dict’s. If the content of the dicts is fairly simple (numbers, tuples, lists, or nested dictionaries thereof), it can be converted into a pandas DataFrame using the to_dataframe method on the callback_data attribute. This may make subsequent analysis easier.

summary.callback_data.to_dataframe("daily")

summary.callback_data.to_dataframe("end_sample")

Users are free to process this callback data now however they like, with typical Python tools: analyze, visualize, interpret, etc.

# Visualize revenue difference between carriers across booking curve

import altair as alt

df = summary.callback_data.to_dataframe("daily").eval("DIFF = AL1 - AL2").query("sample < 100 and trial == 0")

alt.Chart(df).mark_line().encode(
    x=alt.X("days_prior", scale=alt.Scale(reverse=True)),
    y="DIFF",
    color="sample:N",
)

# Visualize "nogo" passengers over time, by market and segment

nogo = (
    summary.callback_data.to_dataframe("daily")
    .set_index(["days_prior", "sample"])
    .drop(columns=["trial", "AL1", "AL2"])
)
nogo.columns = pd.MultiIndex.from_tuples(nogo.columns.str.split(".").to_list())
nogo.columns.names = ["nogo", "orig", "dest", "segment"]
nogo = nogo.stack([1, 2, 3], future_stack=True).dropna().reset_index()

mean_nogo = nogo.groupby(["days_prior", "orig", "dest", "segment"]).nogo.mean().reset_index()
mean_nogo["market"] = mean_nogo.orig + "-" + mean_nogo.dest

alt.Chart(mean_nogo).mark_line().encode(
    x=alt.X("days_prior", scale=alt.Scale(reverse=True)),
    y="nogo",
    color="segment:N",
    strokeWidth="market:N",
    strokeDash="market:N",
)

Tracing¶

In addition to the completely flexible callback data storage system, PassengerSim also includes the ability to “trace” certain details of the simulation. Traces are more aggregate than other callbacks, as they will follow the average values of various measures over many samples. Unlike other aggregate measures, they allow for the selection of a limited number of micro-level details, e.g. path forecasts for specific paths, or bid prices on specific legs. This contrasts with the database functionality, which can (relatively) efficiently store this information for all paths or legs. Tracing allows the analyst to probe a simulation for details of interest without becoming bogged down in the recording and storage of massive amounts of data that really isn’t needed.

sim = pax.Simulation(cfg)

Forecast Tracing¶

The path forecast tracing capabilities allows us to record detailed data about a subset of Simulation path forecasts, so we can review them after the simulation. The tracing will summarize the average forecasts for the specific paths, but will not store the sample-by-sample forecast details, which would be an enormous amount of detail that will require large amounts of memory or disk storage and may not be sufficiently useful for analysis. A similar tracing tool is available for leg forecasts as well.

from passengersim.tracers.forecasts import (
    PathForecastTracer,
    fig_path_forecast_dashboard,
)

f_tracer = PathForecastTracer(path_ids=[1, 5, 9])
f_tracer.attach(sim)

Bid Price Tracing¶

Similarly, the bid price tracing feature allows us to record detailed data about a subset of Simulation bid prices. We can record bid prices by leg, path, or both if desired. Each tracer only attaches to the bid prices on a selected subset of individual legs, so as to not overwhelm the simulation with data.

from passengersim.tracers.bid_price import (
    LegBidPriceTracer,
    PathBidPriceTracer,
    fig_leg_bid_prices,
    fig_path_bid_prices,
)

bp_tracer = PathBidPriceTracer(path_ids=[1, 5, 9])
bp_tracer.attach(sim)

leg_bp_tracer = LegBidPriceTracer(leg_ids=[101, 111])
leg_bp_tracer.attach(sim)

Once the tracers have been attached, we run the simulation as normal. The attached tracers will automatically collect and aggregate the relevant data, and attach the tabulated results to the summary outputs.

summary = sim.run()

Task Completed after 11.57 seconds

All the usual summary data remains available for review and analysis.

summary.fig_carrier_revenues()

summary.fig_fare_class_mix()

Dashboards¶

In addition to the usual summary reports, the output summary also provides the data needed to power detailed dashboard visualizations for each of the traced paths or legs.

For the selected paths or legs, we can review a forecast dashboard that shows the forecast mean and std dev for each fare class from each DCP through departure, the mean forecast within each timeframe, as well as history data on yieldable and (if recorded separately) priceable sales, and average closure rates at each DCP. The displayed data in each dashboard is specific to the selected path or leg, but averaged across all relevant (non-burned) samples in the simulation.

fig_path_forecast_dashboard(summary, path_id=1)

fig_path_forecast_dashboard(summary, path_id=5)

fig_path_forecast_dashboard(summary, path_id=9)

The dashboards for bid price tracing are less busy than those for the forecasts, as the bid price

fig_path_bid_prices(summary)

fig_leg_bid_prices(summary)

Tracers in Callback Data¶

All the underlying data for these reports is stored in the summary’s callback_data attribute, if you want to access it to parse or visualize it differently. The forecast tracers contain selected summary statistics by path or leg, days prior, and fare class. These statistics are aggregated across all relevant (i.e. non-burned) samples.

summary.callback_data.selected_path_forecasts

The bid prices data includes the mean and standard deviation of the leg or path bid price, by days prior.

summary.callback_data.leg_bid_prices

Relationship to Callbacks¶

Unlike other callback data, the tracers are not stored by sample day, as that would generally create an overwhelming amount of data to store, and we are typically not interested in that much detail. If we are interested in grabbing and storing path forecast data for individual sample days, we can still do that with the regular callback interface.

sim1 = pax.Simulation(cfg)

@sim1.begin_sample_callback
def grab_forecasts(sim):
    if sim.eng.sample not in [300, 375]:
        return
    return {f"path-{p}": sim.eng.paths.select(path_id=p).get_forecast_data() for p in [1, 5, 9]}

summary1 = sim1.run()

Task Completed after 9.27 seconds

When run like this, we capture not the average path forecast over the simulation, but rather the exact path forecast for the selected paths (1 and 9) at sample days 300 and 375. The data is stored in the callback_data.begin_sample attribute:

summary1.callback_data.begin_sample

[{'trial': 0,
  'sample': 300,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x11dca03e0>,
  'path-5': <passengersim_core.forecast_tools.ForecastData at 0x11e38c740>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x11e38e000>},
 {'trial': 0,
  'sample': 375,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x11e3ac0b0>,
  'path-5': <passengersim_core.forecast_tools.ForecastData at 0x11e3acbc0>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x11e3ae0f0>},
 {'trial': 1,
  'sample': 300,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x11e3afbc0>,
  'path-5': <passengersim_core.forecast_tools.ForecastData at 0x11e3e8ef0>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x11dd7edb0>},
 {'trial': 1,
  'sample': 375,
  'path-1': <passengersim_core.forecast_tools.ForecastData at 0x11dd7dd90>,
  'path-5': <passengersim_core.forecast_tools.ForecastData at 0x11dd7c770>,
  'path-9': <passengersim_core.forecast_tools.ForecastData at 0x11dda6de0>}]

We can review the details of each specific forecast by accessing the dashboard visualization.

summary1.callback_data.begin_sample[0]["path-1"].dashboard()

We can also access individual sub-tables of forecast data as pandas DataFrames, to manipulate or visualize as we like.

summary1.callback_data.begin_sample[0]["path-1"].history_sold_yieldable

summary1.callback_data.begin_sample[0]["path-1"].mean_in_timeframe