Competitive Pricing in Airline Revenue Management (CARM) with Multi-agent Reinforcement Learning

Peng Wei (George Washington University)

In the 90's operations research (OR) was introduced to the airline industry. The collaboration between OR academics and airline researchers made numerous contributions to many airline applications, including revenue management, network planning, crew scheduling, passenger reaccommodation, operation recovery, etc. Today with the new developments in artificial intelligence (AI), is it time again for the academics and airlines to join forces and bring AI tools such as deep neural networks, reinforcement learning, data-driven modeling and decision making to airline applications? In this talk, Prof. Peng Wei will cover some of his group's recent findings on applying an AI tool called multi-agent reinforcement learning to an airline revenue management problem.

Product and Ancillary Pricing Optimization: Market Share Analytics via Perturbed Utility Model

Changchun Liu (National University of Singapore)

Consider a firm that sells some primary and ancillary products (services) to heterogeneous customers. The challenge is to determine the prices for all the products and services simultaneously, to optimize profits to the firm. This problem is notoriously difficult as it involves choice over subsets (primary product + ancillary services). We consider random utility model for customers' choice problem, and show that the choice model can be reformulated into a perturbed utility model (PUM) over the convex hull of the feasible solutions. Furthermore, we demonstrate how we can obtain a good approximation to the PUM using an additive perturbed utility model (APUM). This allows us to establish a set of closed-form relationships between prices, expected market shares, and interestingly, expected slacks in the constraint matrix of the customer choice problem. This opens up a new way to calibrate the APUM using market share shift information obtained from varying the prices of the products and services. Using piecewise linear approximation, we show that the resulting data-driven pricing problem can be solved as mixed integer linear programs. We show further that the constant markup pricing strategy is within a logarithmic factor of the optimal revenue in our framework, and use this strategy as one of the benchmarks to calibrate the performance of our method. Extensive experimental results demonstrate the superiority of our approach to the state-of-the-art benchmark methods. We finally showcase how this approach can be extended to address competition, and discuss how to solve the quantity discount problem under this framework.

Market-Based and Policy-Based Conditional Demand Forecasting Method

Tim Yuxuan Lu (Massachusetts Institute of Technology)

Price-demand forecasting in competitive markets with undifferentiated fare structures is challenging for airlines. First, demands for each fare class are conditional on fare class availabilities without sufficient differentiation. Second, competitors’ fare class availability also affects demand for an airline’s own offerings. The author presents a novel forecasting philosophy that generates market-based and policy-based fare class demand forecasts, accounting for actual (current) and predicted (future) own airline and competitor availability (policy). These conditional demand estimates can to allow for dynamic optimization of fare adjustment closures according to forecast demand levels and competitors’ fare class availabilities.

Choice Based Revenue Management Systems Simulation

Mitsuyoshi Fukushi (Pontificia Universidad Católica de Chile)

We discuss and test the implementation of a dynamic air transport market simulator, designed to analyze RM systems. The simulator replicates the behavior of passengers that book seats offered in multiple flights by different airlines. We use discrete choice models to replicate the demand behavior, accounting for preferences and decision rule heterogeneity, and including a temporal evolution of the preference throughout the selling horizon. To replicate the supply behavior, a number of airlines modify the price and quantity of different fare classes offered in each flight, using a variety of RM forecasting, un-constraining, and optimization techniques. The simulator allows analysts to study the economic benefit of RM systems under predefined assumptions in an artificial and controlled environment. This increases the benefits obtained by the correct selection of context-appropriate RM systems and the likelihood of successfully implementing new and complex systems. We test and showcase the simulator performance, studying the entrance of a new airline in a competitive context. We generate, implement and evaluate different RM strategies in response to the introduction of new competition, and discuss the results, highlighting the interpretability and accuracy of the proposed framework.

Revenue management with multiple flexible products: downgradables, upgradables and callables

Dhandabani S, R K Amit (Indian Institute of Technology Madras)

Consider an airline operating in an origin-destination pair that has two cabin classes, business and economy. To deal with involuntary denied boarding due to economy class overbooking, airline sells upgradables and callables, which endow the seller with much-needed freedom in planning allocation decisions when the booking process materializes. In addition to these flexibles, we propose to introduce downgradables (product that gives the airline to downgrade the ticket holder from business to economy class) that allow the airline to overbook business class and act as a mitigation technique in the event of excess business show-ups. Formulating it as a dynamic programming problem, we use the multimodularity concepts to show the existence of nested optimal booking policy of threshold form. We derive the necessary conditions for downgradables to be beneficial and highlight the useful, novel insights for a better understanding while including downgradables to their current cartel of products.

Total revenue optimization approaches with the Markov chain choice model

Kevin Wang (Massachusetts Institute of Technology)

The Markov chain choice model (MCCM) has been proposed to construct and price airline ancillary bundle offers that maximize ancillary revenue in the context of dynamic offer creation. However, research also suggests that focusing on maximizing ancillary revenue could consequently reduce an airline’s flight ticket revenue, if customers take ancillary prices into account when choosing their flight. With the goal of total revenue optimization, we propose two heuristic approaches that incorporate a consideration of flight choice into the MCCM. The first approach results in dynamically adjusted ancillary prices depending on a fixed flight price or fare class. The second approach integrates the MCCM as a dynamic pricing engine in the revenue management system and modifies the price of the flight itself. We also discuss potential advantages and disadvantages of the two approaches for airlines. Our results deliver insights that are not only relevant for future dynamic offer creation implementations, but also potentially for airline ancillary and fare family pricing strategies within the current distribution capabilities.

Continuous is the new black!

Burak Ozdaryal, Anubhav Jain, Indrajit Chatterjee, Bazyli Mikolaj Szymanski (United Airlines)

While the airline industry expands its use of NDC, which lays the groundwork for continuous pricing, forecasting and optimization systems still tend to use a small number of discrete fares (filed via fare classes) in each market. This can cause the systems to miss the optimal price, which may be between the fares considered. Rather than using a finer set of discrete fares (continuous forecasting), which would take much more time and computing resources, we would like to review a proof-of-concept design for a continuous optimizer that is not dependent on discrete fares/classes. This concept doesn’t just improve runtimes -- it can also provide flexibility to expand and jointly optimize fares of competing itineraries in a market. 

Predicting Airline Passenger No-Show with Machine Learning & PNR Features for Revenue Optimization

Cindy Yao, Alan Regis (Air Canada)

Airlines determine the number of seats by which to oversell on each flight, balancing between minimizing expected spoiled seats, minimizing expected denied boarding compensation, and maximizing level of service provided. The main input to this optimization is a forecast of passenger no-show rates, the accuracy of which is key. We developed a machine learning forecasting algorithm that adapts to seasonality and fast-changing data and integrates novel PNR-based features to deliver more accurate passenger no-show predictions. We have since deployed the model on a variety of markets to guide oversell strategies and increase revenues.

Machine Learning meets Causal Inference: A Hybrid Framework for Price Sensitivity Estimation

Ravi Kumar, Shahin Boluki (PROS, Inc.)

Many sellers are interested in improving their pricing decisions by dynamically adjusting prices of their products based on unique product features and other relevant information available at the time of request. Automated dynamic pricing systems to enable such functionality typically require estimates of customers' price-sensitivity using historical sales transaction data. In many industries where sales channels are not fully controlled by the firm, access to good quality loss information is challenging. In the absence of loss information, one can estimate the price-sensitivity by learning price-demand relationships (demand response modeling) with aggregated sales for a product in given time period e.g., daily sales. Estimation of price-sensitivity parameters via demand response modeling lies in the realm of Causal Inference (not prediction) and construction of estimators robust to confounders and model mis-specifications remains a challenge. Modern machine learning (ML) approaches like neural networks and gradient boosting, despite their high predictive power, do not easily lend themselves to constructing an interpretable framework for the inference task related price elasticity estimation. In this work we propose a hybrid framework that leverages the synergy between predictive power of modern Machine Learning (ML) approaches and interpretable semi-parametric models for robust price-elasticity estimation. We show the performance of these methods via simulation studies.

Cargo Show-Up Rate Prediction

Gurveen Kaur (Sabre Travel Technologies India Pvt. Ltd.)

Cargo service primarily serves B2B customers (freight forwarders), offering to transport cargo on the freight network. Vast majority of the shippers are serviced through Freight Forwarding Agents like DHL, with very few shippers serviced as direct for personal cargo. B2B Freight Forwarders make up majority of cargo airline business. Since the Cargo Bookings are postpaid, currently there are no penalties for Booking Cancellations which causes No-Shows or last-minute cancellations resulting in revenue loss. Apart from very specific contracts, there is no major penalty for customers. The airline cargo observation is 15-30% no-shows / cancellations between D-3 and D-0 on average for the whole network. The approach developed jointly with Etihad (EY) Cargo team is a regression learning model to identify risky Bookings which will be a potential No-Show or can be under delivering compared to what has been the booked volume or weight. This model uses EY's booking snapshot data which is available at an AWB-market level for flights starting from D-14 until D+7, where D is the date of departure and recommendations are shared with end users on each day before departure. Benefits from the model are recognized in load factor improvement, revenue dilution avoidance and improvement of Cargo delivery.

Continuous Markov Chain Choice for Schedules

Jonas Rauch (PROS, Inc.)

We present a parametric, continuous version of the Markov Chain Choice (MCC) model specifically suited for choice among departure times in a schedule. The model is derived as the continuous limit of the discrete Markov Chain Choice model on a one-dimensional product space. Customer choice behavior is represented by a continuous-time Markov process, consisting of two interdependent stochastic processes: A diffusion process that models the customer’s transition through the space of candidate products, and a jump process that represents a transition to the no-purchase option. We give explicit analytic expressions for the choice probabilities assuming piece-wise constant model parameters. The continuous MCC model combines the advantages of the discrete MCC model (realistic recapture behavior while still being numerically tractable), with the advantages of parametric models (ability to extrapolate and predict choice probabilities for previously unobserved products). All model parameters have an intuitive interpretation, which allows experts to validate the model and gain managerial insights. These properties make the model particularly attractive for practical applications in airline revenue management or network planning, or in similar contexts in other industries where schedules change frequently. Furthermore, many known extensions of the discrete MCC model directly apply to our model as well.

From Passive to Active Learning: Improving forecast quality and revenue through price experimentation

Mike Wittman and Thomas Fiig (Amadeus IT Group)

A typical revenue management system (RMS) operates in a greedy passive learning mode – it estimates a demand forecast from historic data and always offers the perceived revenue maximizing price. However, this framework can struggle when a lack of price variation in the historic database or a change in demand behavior results in large uncertainty on the estimated forecast parameters. In this presentation, we propose an active learning strategy for RMS that aims to solve a dual control problem: optimizing revenue while simultaneously gaining knowledge about the environment. We show how price exploration – offering potentially “sub-optimal” prices in certain situations – can improve the forecast and lead to long-term revenue gains despite a potential risk of short-term revenue loss. We describe how to design a revenue-optimal active learning strategy that exploits when forecast accuracy is high and explores when forecast accuracy is low. Finally, in collaboration with an airline and using actual data, we evaluate the opportunity for active learning and discuss how this strategy can be implemented in practice.

Are You Ready to Skip Demand Forecasting? A Neural Network based Adaptive Approach to Revenue Management

Ezgi Eren (PROS, Inc.)

Since the start of the pandemic, the need to have more adaptive approaches in Revenue Management has become more emphasized. As the historical data have become less reliable, there is also increasing need for robust approaches with less rigidity in data requirements and assumptions. We present Direct Adaptive Neural Network Based Revenue Management (DiANNe) that addresses both these needs. DiANNe is a non-conventional approach to Revenue Management as it skips demand forecasting and utilizes machine learning for generating bid prices. As a result, it is typically a lighter implementation with much less strict data requirements. It has an adaptive nature that handles volatility well and is robust to everchanging customer behavior. We highlight results from simulations where we tested DiANNe’s response to significant shocks and shifts in the incoming data, which showcases its adaptive nature.

Multi-itinerary market adaptive dynamic pricing

Octavian Oancea (Sabre)

Traditionally, the inventory control policies generated by a revenue management optimizer are based on a fixed set of price points, expected demand to come, capacity left to sell, and time left until departure. These control policies are determined without knowledge of the competitors’ market positioning. Such knowledge can be used dynamically to improve revenues by identifying an optimized markup to be applied across all itineraries offered by an airline on a given market. The output is consisted of price adjustments based on the optimized markup, using a choice model calibrated on the targeted customer trip purpose segments. An overview of this multi-itinerary market adaptive dynamic pricing model will be presented, as well as the various challenges faced in tackling real airline data.

Come One Come All: Sharing Data Across Airlines is a Positive-Sum Game

Ross Winegar (PROS, Inc.)

What happens when airlines share data? Most RM science models are trained and deployed in a silo for each airline individually using only their unique passenger data, allowing them to only react to changes in their own markets and passenger base. In the midst of the pandemic different markets opened and closed at different times and different countries allowed certain citizens to travel back and forth. To generate global predictions for recovery PROS built a random forest model combining data from close to 2 dozen carriers so that changes in one part of the world could be inferred to predict other parts. This gave an opportunity to measure the improved accuracy in ML modelling from joining airline data together as opposed to running each airline in a silo, with a measured average RMSE improvement of 8.16%.

Fare Structure Optimization using Agent Based Modeling and Bayesian Optimization

Javier Jimenez (Airnguru)

The optimization of fare structures in the airline industry has been a headache for many pricing teams over the last decade. New methodologies can be implemented to solve the optimization problem thanks to more data available and newer technology to handle such data. We propose using Agent-Based Modeling simulation techniques in conjunction with Bayesian Optimization to optimize fare structures through the convergence of maximum net expected revenue. We observe that such a method presents promising results, but that needs to be validated in real-life scenarios. We also see opportunities to use other more advanced optimization methods, which could be less transparent (such as deep reinforcement learning) but more efficient in terms of online calculation effort.

A heuristic for incorporating ancillaries into air choice models with personalization

Michal Sznajder, Richard Ratliff (Sabre)

In recent years, airlines have begun selling “branded fares” which are bundles of a travel “right-to-fly” plus assorted amenities (e.g., 1st bag included, extra legroom, priority boarding, extra loyalty miles, etc.). Branded fares allow airlines to further differentiate their products and have proven popular with customers. Previously, air choice models in use at Sabre were limited to price and itinerary schedule attributes only; they did not consider the value-added utility of amenities included in airline branded fares. In this talk, we describe a heuristic (based on hedonic regression) to extend air choice models to include the additional customer utility arising from airline branded fare ancillaries. When applied to shopping results that include branded fares, this new approach led to 10% improvement in air choice model prediction accuracy. We will also present a new choice-model accuracy metric (TOPN%) that allows comparison of results across samples with different assortment sizes. Furthermore, we show how to extend this choice model with ancillaries to include personalization by customer segment.