Firehose¶
In addition to all the pre-processed and RM data that is a available from PassengerSim, if you really (really, really) want to see the details, you can connect to a “firehose” data stream, which outputs virtually everything going on inside the simulation. This is generally a lot of data, and you will not want to do this when running PassengerSim at scale. Even for the simple 3 market example model, firehose outputs can easily output over a gigabyte of data.
import io
import passengersim as pax
pax.versions()
passengersim 0.80
passengersim.core 0.80
cfg = pax.Config.from_yaml(pax.demo_network("3MKT"))
cfg.simulation_controls.num_samples = 51
cfg.simulation_controls.burn_samples = 50
cfg.simulation_controls.num_trials = 1
cfg.db = None
cfg.outputs.reports.clear()
cfg.carriers.AL1.rm_system = "P"
cfg.simulation_controls.show_progress_bar = False
sim = pax.Simulation(cfg)
It is possible to attach a firehose to the simulation at any point, including before you even begin. For example, here we’ll attach the “orders” data stream to stdout, which will print the output right here in our Python interpreter.
sim.attach_firehose("orders", "stdout")
trial, sample, time, event_num, num_pax, booking_class, chosen_as_priceable, price, wtp, orig, dest, segment, path_id, legs_with_remaining_capacity
You can see here, when you attach to the “orders” firehose, it writes a header line. In the Jupyter notebook context, this header shows up when
Since the firehose is going to overwhelm us with data, we’ll make a callback to turn it off almost immediately.
@sim.daily_callback
def omg(sim, days_prior):
if sim.eng.sample == 0 and days_prior == 60:
sim.attach_firehose("orders", None)
Since the firehose sends out a lot of data, it’s unlikely we’d actually want to see it in “stdout” (or “stderr”). Fortunately, we can also give a filename (any string other than those two) or a writable file-like object. Let’s hook up to the 50th sample to collect all the orders.
orders_50 = io.StringIO()
@sim.begin_sample_callback
def please_may_i_have_some_more(sim):
if sim.eng.sample == 50:
sim.attach_firehose("orders", orders_50)
@sim.end_sample_callback
def gee_thanks(sim):
if sim.eng.sample == 50:
sim.attach_firehose("orders", None)
Now we’ll run the model.
out = sim.run()
0,0,2019-12-29 10:17:42 UTC,6,1,Y4,0,175,350.148,ORD,LAX,leisure,6,AL1:112 ORD-LAX /120
0,0,2019-12-29 10:54:36 UTC,7,1,Y3,0,150,176.142,BOS,ORD,leisure,4,AL2:202 BOS-ORD /100
0,0,2019-12-29 14:51:05 UTC,8,1,Y4,0,250,485.673,BOS,LAX,leisure,12,AL2:202 BOS-ORD /99 | AL2:212 ORD-LAX /120
0,0,2019-12-29 16:22:50 UTC,9,1,Y4,0,175,188.368,ORD,LAX,leisure,5,AL1:111 ORD-LAX /120
0,0,2019-12-29 19:23:13 UTC,10,1,Y4,0,175,468.663,ORD,LAX,leisure,5,AL1:111 ORD-LAX /119
0,0,2019-12-29 19:49:09 UTC,11,1,Y4,0,175,267.108,ORD,LAX,leisure,8,AL2:212 ORD-LAX /119
0,0,2019-12-29 20:06:29 UTC,12,1,Y5,1,100,118.05,BOS,ORD,leisure,2,AL1:102 BOS-ORD /100
0,0,2019-12-30 03:56:09 UTC,13,1,Y4,0,250,462.75,BOS,LAX,leisure,10,AL1:102 BOS-ORD /99 | AL1:112 ORD-LAX /119
0,0,2019-12-30 04:24:36 UTC,14,1,Y4,0,125,270.7,BOS,ORD,leisure,1,AL1:101 BOS-ORD /100
0,0,2019-12-30 08:04:40 UTC,19,1,Y5,1,200,263.529,BOS,LAX,leisure,12,AL2:202 BOS-ORD /98 | AL2:212 ORD-LAX /118
0,0,2019-12-30 11:34:11 UTC,20,1,Y4,0,175,244.473,ORD,LAX,leisure,6,AL1:112 ORD-LAX /118
0,0,2019-12-30 11:35:28 UTC,21,1,Y4,0,125,176.7,BOS,ORD,leisure,4,AL2:202 BOS-ORD /97
0,0,2019-12-30 13:07:07 UTC,22,1,Y4,0,175,229.638,ORD,LAX,leisure,5,AL1:111 ORD-LAX /118
0,0,2019-12-30 16:19:04 UTC,23,1,Y5,1,200,244.303,BOS,LAX,leisure,9,AL1:101 BOS-ORD /99 | AL1:111 ORD-LAX /117
0,0,2019-12-30 21:15:08 UTC,24,1,Y4,0,175,182.403,ORD,LAX,leisure,7,AL2:211 ORD-LAX /120
0,0,2019-12-30 21:40:02 UTC,25,1,Y4,0,175,406.6,ORD,LAX,leisure,6,AL1:112 ORD-LAX /117
0,0,2019-12-30 22:54:10 UTC,26,1,Y4,0,250,504.26,BOS,LAX,leisure,11,AL2:201 BOS-ORD /100 | AL2:211 ORD-LAX /119
0,0,2019-12-30 23:20:05 UTC,27,1,Y5,1,150,154.796,ORD,LAX,leisure,6,AL1:112 ORD-LAX /116
0,0,2019-12-31 00:02:25 UTC,28,1,Y4,0,175,200.877,ORD,LAX,leisure,5,AL1:111 ORD-LAX /116
0,0,2019-12-31 06:52:32 UTC,33,1,Y0,0,500,2081.91,ORD,LAX,business,5,AL1:111 ORD-LAX /115
0,0,2019-12-31 09:52:12 UTC,34,1,Y4,0,125,327.626,BOS,ORD,leisure,1,AL1:101 BOS-ORD /98
0,0,2019-12-31 10:38:30 UTC,35,1,Y4,0,125,223.988,BOS,ORD,leisure,2,AL1:102 BOS-ORD /98
0,0,2019-12-31 10:57:44 UTC,36,1,Y4,0,125,193.047,BOS,ORD,leisure,2,AL1:102 BOS-ORD /97
0,0,2019-12-31 11:29:56 UTC,37,1,Y4,0,125,461.513,BOS,ORD,leisure,4,AL2:202 BOS-ORD /96
0,0,2019-12-31 12:40:15 UTC,38,1,Y0,0,500,2764.46,ORD,LAX,business,7,AL2:211 ORD-LAX /118
0,0,2019-12-31 15:44:52 UTC,39,1,Y4,0,250,268.391,BOS,LAX,leisure,12,AL2:202 BOS-ORD /95 | AL2:212 ORD-LAX /117
0,0,2019-12-31 21:52:58 UTC,40,1,Y4,0,250,320.524,BOS,LAX,leisure,9,AL1:101 BOS-ORD /97 | AL1:111 ORD-LAX /114
0,0,2020-01-01 03:13:06 UTC,41,1,Y4,0,175,216.342,ORD,LAX,leisure,6,AL1:112 ORD-LAX /115
0,0,2020-01-01 05:01:34 UTC,42,1,Y4,0,125,196.99,BOS,ORD,leisure,2,AL1:102 BOS-ORD /96
We got a bunch of orders data from the very first sample in our stdout.
We also now have access to order from the 50th sample… they were not printed to our screen but they are available in our pseudo-file.
len(orders_50.getvalue())
68258
We got a fair bit of data from our little model! Let’s take a peek:
data = orders_50.getvalue().splitlines()
print("\n".join(data[:10] + ["..."] + data[-18:]))
trial, sample, time, event_num, num_pax, booking_class, chosen_as_priceable, price, wtp, orig, dest, segment, path_id, legs_with_remaining_capacity
0,50,2019-12-29 07:53:28 UTC,45838,1,Y4,0,175,210.838,ORD,LAX,leisure,6,AL1:112 ORD-LAX /120
0,50,2019-12-29 12:08:28 UTC,45839,1,Y5,1,100,111.864,BOS,ORD,leisure,3,AL2:201 BOS-ORD /100
0,50,2019-12-29 12:11:06 UTC,45840,1,Y5,1,200,223.583,BOS,LAX,leisure,12,AL2:202 BOS-ORD /100 | AL2:212 ORD-LAX /120
0,50,2019-12-29 12:17:58 UTC,45841,1,Y4,0,175,190.735,ORD,LAX,leisure,5,AL1:111 ORD-LAX /120
0,50,2019-12-29 12:33:31 UTC,45842,1,Y2,0,300,465.261,ORD,LAX,business,8,AL2:212 ORD-LAX /119
0,50,2019-12-29 15:28:40 UTC,45843,1,Y4,0,175,175.848,ORD,LAX,leisure,8,AL2:212 ORD-LAX /118
0,50,2019-12-29 17:49:37 UTC,45844,1,Y4,0,175,247.363,ORD,LAX,leisure,7,AL2:211 ORD-LAX /120
0,50,2019-12-29 18:41:54 UTC,45845,1,Y1,0,625,939.322,BOS,LAX,business,9,AL1:101 BOS-ORD /100 | AL1:111 ORD-LAX /119
0,50,2019-12-30 00:42:22 UTC,45846,1,Y4,0,250,1280.88,BOS,LAX,leisure,12,AL2:202 BOS-ORD /99 | AL2:212 ORD-LAX /117
...
0,50,2020-02-29 17:47:28 UTC,46775,1,XX,0,0,379.912,ORD,LAX,business,-1,
0,50,2020-02-29 17:47:31 UTC,46776,1,XX,0,0,167.78,ORD,LAX,leisure,-1,
0,50,2020-02-29 21:12:49 UTC,46777,1,Y1,1,300,595.483,BOS,ORD,business,4,AL2:202 BOS-ORD /24
0,50,2020-02-29 21:24:18 UTC,46778,1,XX,0,0,533.805,BOS,LAX,business,-1,
0,50,2020-02-29 22:34:29 UTC,46779,1,Y1,1,400,499.186,ORD,LAX,business,8,AL2:212 ORD-LAX /3
0,50,2020-02-29 22:56:12 UTC,46780,1,Y0,0,750,1209.57,BOS,LAX,business,12,AL2:202 BOS-ORD /23 | AL2:212 ORD-LAX /2
0,50,2020-02-29 22:56:56 UTC,46781,1,Y1,1,300,436.905,BOS,ORD,business,2,AL1:102 BOS-ORD /9
0,50,2020-02-29 23:09:36 UTC,46782,1,Y0,0,500,1355.06,ORD,LAX,business,8,AL2:212 ORD-LAX /1
0,50,2020-02-29 23:14:52 UTC,46783,1,Y0,0,500,888.88,ORD,LAX,business,6,AL1:112 ORD-LAX /3
0,50,2020-02-29 23:52:43 UTC,46784,1,XX,0,0,476.928,ORD,LAX,business,-1,
0,50,2020-03-01 00:26:38 UTC,46785,1,XX,0,0,351.117,ORD,LAX,leisure,-1,
0,50,2020-03-01 02:11:03 UTC,46786,1,XX,0,0,313.297,BOS,LAX,leisure,-1,
0,50,2020-03-01 02:59:34 UTC,46787,1,XX,0,0,537.713,BOS,LAX,business,-1,
0,50,2020-03-01 03:02:40 UTC,46788,1,XX,0,0,529.131,BOS,LAX,business,-1,
0,50,2020-03-01 03:32:59 UTC,46789,1,Y0,1,500,515.704,ORD,LAX,business,6,AL1:112 ORD-LAX /2
0,50,2020-03-01 03:33:33 UTC,46790,1,XX,0,0,-1,ORD,LAX,leisure,-1,
0,50,2020-03-01 04:13:47 UTC,46791,1,Y1,1,625,1571.12,BOS,LAX,business,10,AL1:102 BOS-ORD /8 | AL1:112 ORD-LAX /1
0,50,2020-03-01 05:20:23 UTC,46792,1,XX,0,0,-1,ORD,LAX,leisure,-1,
We can see the expected trends: at the beginning, there’s plenty of capacity, and we see and accept mostly leisure passengers. At the end of the order stream, we are mostly turning away customers, although a few business customers with high willingness to pay are able to make orders.
We can feed this raw CSV output back into a tool that can process it if we like.
import pandas as pd
# Reset the StringIO cursor to the beginning so we can read it again
orders_50.seek(0)
pd.read_csv(orders_50)
| trial | sample | time | event_num | num_pax | booking_class | chosen_as_priceable | price | wtp | orig | dest | segment | path_id | legs_with_remaining_capacity | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 50 | 2019-12-29 07:53:28 UTC | 45838 | 1 | Y4 | 0 | 175 | 210.838 | ORD | LAX | leisure | 6 | AL1:112 ORD-LAX /120 |
| 1 | 0 | 50 | 2019-12-29 12:08:28 UTC | 45839 | 1 | Y5 | 1 | 100 | 111.864 | BOS | ORD | leisure | 3 | AL2:201 BOS-ORD /100 |
| 2 | 0 | 50 | 2019-12-29 12:11:06 UTC | 45840 | 1 | Y5 | 1 | 200 | 223.583 | BOS | LAX | leisure | 12 | AL2:202 BOS-ORD /100 | AL2:212 ORD-LAX /120 |
| 3 | 0 | 50 | 2019-12-29 12:17:58 UTC | 45841 | 1 | Y4 | 0 | 175 | 190.735 | ORD | LAX | leisure | 5 | AL1:111 ORD-LAX /120 |
| 4 | 0 | 50 | 2019-12-29 12:33:31 UTC | 45842 | 1 | Y2 | 0 | 300 | 465.261 | ORD | LAX | business | 8 | AL2:212 ORD-LAX /119 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 702 | 0 | 50 | 2020-03-01 03:02:40 UTC | 46788 | 1 | XX | 0 | 0 | 529.131 | BOS | LAX | business | -1 | NaN |
| 703 | 0 | 50 | 2020-03-01 03:32:59 UTC | 46789 | 1 | Y0 | 1 | 500 | 515.704 | ORD | LAX | business | 6 | AL1:112 ORD-LAX /2 |
| 704 | 0 | 50 | 2020-03-01 03:33:33 UTC | 46790 | 1 | XX | 0 | 0 | -1.000 | ORD | LAX | leisure | -1 | NaN |
| 705 | 0 | 50 | 2020-03-01 04:13:47 UTC | 46791 | 1 | Y1 | 1 | 625 | 1571.120 | BOS | LAX | business | 10 | AL1:102 BOS-ORD /8 | AL1:112 ORD-LAX /1 |
| 706 | 0 | 50 | 2020-03-01 05:20:23 UTC | 46792 | 1 | XX | 0 | 0 | -1.000 | ORD | LAX | leisure | -1 | NaN |
707 rows × 14 columns